This is a request for a 3-year generic clearance for the Office of Juvenile Justice and Delinquency Prevention (OJJDP) that will allow it to develop, test, and improve its survey and data collection instruments and methodologies. The procedures utilized to this effect include, but are not limited to, tests of various types of survey and data collection operations, focus groups, cognitive laboratory activities, pilot testing, exploratory interviews, experiments with questionnaire design, and usability testing of electronic data collection instruments.
OJJDP is requesting the generic clearance in order to test new methodologies for surveys and data collection activities. OJJDP is responsible for a number of high profile, complex national data collections to provide policymakers, practitioners, and the public with important information, including in the areas of child victimization and youth involvement with the juvenile justice system. OJJDP has been coordinating closely with the Bureau of Justice Statistics (BJS) to enhance its data collection efforts, both self-report and establishment surveys, to ensure they meet the highest and most efficient standards for design and data collection methodologies. We believe the generic clearance will be a helpful vehicle for evaluating questionnaires and various other data collection procedures related to new or changing data collections.
Prior to this request, OJJDP largely relied on expert review and/or convenience samples of 9 or fewer persons to provide input and feedback on survey design and data collection methodologies. This approach does not meet OJJDP’s needs to develop and implement more rigorous testing procedures. Relatedly, seeking full OMB clearance prior to testing instruments will cause delays to OJJDP achieving its timeliness objectives for collections. Reliance on 9 or fewer persons does not provide a basis for conducting any type of test and the data from 9 or fewer have no generalizability. The information collected via these mechanisms, while still helpful, was nonetheless somewhat limited in its ability to detect and diagnose problems with the instruments and the procedures being tested.
The generic testing clearance will allow OJJDP to take advantage of a variety of methods that are useful for identifying questionnaire/assessment and procedural problems, suggesting solutions, and measuring the relative effectiveness of alternative solutions. Through the use of these techniques, when employed routinely in the testing phase of OJJDP data collections, questionnaires and assessments can be simplified for respondents, respondent burden can be reduced, procedures for the collection of administrative data can be streamlined, and the quality of the questionnaires and assessments used in continuing and one-time surveys and assessments can be improved. Thus an increase in the quality of the data collected can be achieved as well.
OJJDP is requesting a three-year generic clearance for pretesting, during which OJJDP will provide periodic reports on pretesting activities. The pretesting activities conducted under this generic clearance will be for development work only. These development activities will include such things as investigation of item types, research on the availability and quality of administrative data from state and local justice agencies, small scale tests to test appropriate access and retrieval methods for various types of administrative data, research about mode of administration (telephone, paper and pencil, computer-based, mail-out and mail-in, etc.), methodology of questionnaires and assessments, and testing of items. Activities covered under this generic clearance will not include field testing of a full-scale program implementation protocol.
This clearance package is intended to serve as a request for generic clearance. In this document we have provided a description of the scope of possible activities that might be covered under this clearance. The requested clearance is important to OJJDP’s use of pretesting activities, because of the length of time required to plan the activities. This generic clearance will go through the usual two Federal Register Review periods. Subsequent to these review periods, OJJDP requests that OMB review then comment on or clear proposed studies in a two-week period with no 30-day Federal Register Notice period required under the generic clearance. This clearance is similar to the testing clearances held by BJS, the Census Bureau, the Bureau of Labor Statistics, the National Center for Education Statistics, and the National Center for Science and Engineering Statistics.
The specific methods proposed for coverage by this clearance are described below. Also outlined are the procedures OJJDP plans to put in place for keeping OMB informed about the identity of the surveys and the nature of the research activities being conducted.
The methods proposed for use in questionnaire and assessment development are as follows:
Pilot testing. For the purposes of this clearance, we are defining pilot tests as data collection efforts conducted among either purposive or statistically representative samples, for which evaluation of the questionnaire and/or procedures is the main objective. OJJDP will only publish research and development (R&D) and methodological reports on the results of these pilot tests, but will not publish statistical reports or data sets based on the findings. Pilot tests are an essential component of this clearance package because they serve as the vehicle for investigating basic item properties for new or redesigned data collection efforts, such as reliability, validity, and difficulty, as well as feasibility of methods for standardized administration of forms. Under this clearance a variety of surveys will be pretested, and the exact nature of the surveys and the samples is undetermined at present. However, due to the smaller nature of the tests, we expect that some will not involve representative samples. In these cases, samples will basically be convenience samples, which could be limited to specific geographic locations, may involve expired rotation groups of a current survey, are known to have specific aggregate demographic characteristics, etc. The needs of the particular sample will vary based on the content of the survey being tested, but the selection of sample cases will not be completely arbitrary in any instance. Where applicable, pilot testing may also include collecting sample administrative data, which will allow OJJDP to test procedures regarding data procurement as well as comparability of data across sites.
Behavior coding. This method involves applying a standardized coding scheme to the completion of an interview or questionnaire, either by a coder using a tape-recording of the interview or by an in-person observer at the time of the interview. The coding scheme is designed to identify situations that occur during the interview that reflect problems with the questionnaire. For example, if respondents frequently interrupt the interviewer before the question is completed, the question may be too long. If respondents frequently give inadequate answers, this suggests there are some other problems with the question. Quantitative data derived from this type of standardized coding scheme can provide valuable information to identify problem areas in a questionnaire, and can be used as a substitute for or as a complement to the traditional interviewer debriefing.
Interviewer debriefing. This method employs the knowledge of the employees who have the closest contact with the respondents. In conjunction with other methods, we plan to use this method in our field tests to collect information about how interviewers react to the survey instruments, as well as understand problems and pitfalls encountered by interviewers during the interview.
Exploratory interviews. These may be conducted with individuals to understand a topical area and may be used in the very early stages of developing a new survey. It may cover discussions related to administrative records (e.g. what types of records, where, and in what format), subject matter, definitions, etc. Exploratory interviews may also be used to investigate whether sufficient issues are present related to an existing data collection to consider a redesign.
Respondent debriefing questionnaire. In this method, standardized debriefing questionnaires are administered to respondents who have participated in a field test. The debriefing form is administered at the end of the questionnaire being tested, and contains questions that probe to determine how respondents interpret the questions and whether they have problems in completing the survey/questionnaire. This structured approach to debriefing enables quantitative analysis of data from a representative sample of respondents, to learn whether respondents can answer the questions, and whether they interpret them in the manner intended by the questionnaire designers.
Follow-up interviews or re-interviews. This involves re-interviewing or re-assessing a sample of respondents after the completion of a survey or assessment. Responses given in the re-interview are compared with the respondents’ initial responses for consistency between responses. In this way, re-interviews provide data for studies of test–re-test reliability and other measures of data quality. In turn, this information aids in the development of improved, more reliable measures.
Cognitive and usability interviews. This method involves intensive, one-on-one interviews in which the respondent is typically asked to "think aloud" as he or she answers survey questions. A number of different techniques may be involved, including asking respondents to paraphrase questions, probing questions asked to determine how respondents came up with their answers, and so on. The objective is to identify problems of ambiguity or misunderstanding, or other difficulties respondents have answering questions. This is frequently one of the early stages of revising a questionnaire.
Focus groups. This method involves group sessions guided by a moderator, who follows a topical outline containing questions or topics focused on a particular issue, rather than adhering to a standardized questionnaire. Focus groups are useful for surfacing and exploring issues (e.g., confidentiality concerns) which people may feel some hesitation about discussing.
Procedures for Clearance
Before testing activity is undertaken, OJJDP will provide OMB with a memo describing the study to be conducted and a copy of instrumentation and debriefing materials that will be used. Depending on the stage of instrumentation development, this may be a printed questionnaire, a set of prototype items showing each item type to be used and the range of topics to be covered by the questionnaire, or an interview script. When split sample experiments are conducted, either in small group sessions or as part of a field test, the different versions of the questionnaires to be used will be provided. For a test of alternative procedures, the description and rationale for the procedures will be submitted. A brief description of the planned field activity will also be provided. OJJDP requests that OMB raise comments on substantive issues within 10 working days of receipt.
Data collections that will be the focus of requested generic clearances will be permitted under authorizing legislation. In most cases, data collection activities conducted by OJJDP will be authorized under the Juvenile Justice and Delinquency Prevention Act of 1974, as amended (42 USC, Section 5661) see Appendix A. At this time, it is not known whether other titles will be referenced for specific projects, as we do not know all of the data collections that will be pretested during the course of the clearance. The authorizing statute will be specified in each requested clearance.
2. Needs and Uses
The information collected under a generic clearance will be used by staff from OJJDP to evaluate and improve the quality of the data in the surveys and assessments that are ultimately conducted. None of the data collected under this clearance will be published for its own sake.
Because the data collections being tested under this clearance are still in the process of development, the data that result from these collections are not considered official statistics of OJJDP or other Federal agencies. Data will not be made public, except when they are included in research reports prepared for sponsors inside and outside of OJJDP. The results may also be prepared for presentations related to data collection methodology at professional meetings or publications in professional journals.
OJJDP anticipates conducting testing work under this Generic Information Clearance for a number of projects, including but not limited to the following:
Juvenile Residential Facility Census (JRFC): The JRFC collects data on how facilities operate and the services they provide. It includes questions on facility ownership and operation, security, capacity and crowding, and injuries and deaths in custody. The JRFC also collects supplementary information each year on specific services, such as mental and physical health, substance abuse, and education. OJJDP will pursue development work for the JRFC (OMB No. 1121-0219) through (1) retrospective cognitive debriefings of data providers in an effort to identify the necessary changes to the survey instrument to enhance data quality, reduce burden, and increase efficiencies for the annual collections, (2) development and cognitive testing of new items and modifications of past items in the core instrument and supplemental modules, and (3) pilot testing of new and modified items to determine respondent capacity to report survey items and estimate the burden associated with responding to the questionnaire. Estimated burden hours needed for cognitive debriefings, facility questionnaire review and testing, and for pilot testing are 300 hours.
Census of Juveniles in Residential Placement (CJRP): The CJRP asks all juvenile residential facilities in the United States to describe each person younger than 21 assigned a bed in the facility on the census date because of an offense. Facilities report individual-level information on gender, date of birth, race, placement authority, most serious offense charged, court adjudication status, and admission date. OJJDP will pursue development work for the CJRP (OMB No. 1121-0218) through (1) retrospective cognitive debriefings of data providers in an effort to identify the necessary changes to the survey instrument to enhance data quality, reduce burden, and increase efficiencies for the annual collections, (2) determine the feasibility of collecting aggregate information from facilities about admissions, releases, and average length of stay, and (3) pilot testing of new and modified items to estimate the burden associated with responding to the questionnaire. Estimated burden hours needed for cognitive debriefings, questionnaire review and testing, and for pilot testing are 300 hours.
Census of Juveniles on Probation (CJP): The CJP collected (OMB No. 1121-0291) individual-level data about youth on formal probation, including their age, sex, race, residence, offense type, and offense location. OJJDP would like to engage in activities, including cognitive interviews, to determine the feasibility of future administrations and to inform the possible redesign of the CJP questionnaire to identify a core set of data elements that can be used to produce key statistics to track changes in size and composition of the nation’s formal juvenile probation populations. Estimated burden hours needed for cognitive interviews are 100 hours.
OJJDP intends to work with a successful applicant to implement an instrument and data collection protocol to generate national estimates on child victims of stranger abductions (i.e., “stereotypical kidnappings”) known to law enforcement agencies. OJJDP would seek clearance for research in support of this effort to include pilot testing the collection in order to assess the functionality of the instrument, the capacity of respondents to provide the information, and the level of burden associated with reporting the information prior to implementing it on a national scale. The findings will be used to make the necessary changes to the instrument to enhance its functionality, improve the quality of the data collected, and minimize burden prior to national implementation. Estimated burden hours needed for pilot testing the instrument, administration protocols and follow-up procedures are 200 hours.
OJJDP intends to work with a successful applicant to develop and test strategies to collect information from law enforcement agencies on parental abductions and other types missing episodes involving children. OJJDP would seek clearance for developmental research in support of this effort to include exploratory interviews to inform survey development and pilot testing the collection to include cognitive interviews in order to assess the functionality of the instrument, the capacity of respondents to provide the information, and the level of burden associated with reporting the information prior to implementing it on a national scale. The findings will be used to make the necessary changes to the instrument to enhance its functionality, improve the quality of the data collected, and minimize burden prior to national implementation. Estimated burden hours needed for exploratory interviews, pilot testing the administration protocols and follow-up procedures, and for cognitive testing the new instrumentation are 250 hours.
OJJDP intends to survey relevant agencies in states and a sample of counties to assess the feasibility of providing individual-level, administrative data sets regarding dual juvenile justice (JJ) and child welfare (CW) involvement for the Design Study of Dual System Youth. The Design Study of Dual System Youth aims to determine the potential survey design and feasibility to estimate a national incidence rate of dual involvement in the child welfare and juvenile justice systems overall and by pathway. In the landscape survey, an estimated 50 states would be contacted to assess their JJ and CW systems in detail. This would include, but is not limited to the existence of statewide systems, their coverage of the state and of case types, the variables available in the system(s) and their data quality, and the willingness of the state to provide statewide, case-level file extracts or data dumps covering multiple years. A sample of counties would then be contacted to determine the feasibility of sampling counties in jurisdictions without statewide systems. The findings from this landscape survey would be used to inform the potential design of a national study of dual system youth, its feasibility, and potential cost. Estimated burden hours needed for landscape survey of states and counties is 250 hours.
3. Use of Information Technology
When the survey or assessment being pretested employs automated methods for its data collection, the research conducted under this submission will also use automated data collection techniques. This clearance offers OJJDP the opportunity to try innovative technologies that can reduce burden, improve data quality and reliability, and increase the use of information technology.
4. Efforts to Identify Duplication
The research to be requested under a generic clearance will not duplicate any other methodological work being done by OJJDP or other Federal agencies. The purpose of the clearance will be to stimulate additional research which would not be done under other circumstances. The research will involve collaboration with staff from other agencies that are sponsoring surveys conducted by OJJDP, when applicable. The research may also involve joint efforts with staff from other Federal laboratory facilities. All efforts would be collaborative in nature, and no duplication in this area is anticipated.
To the maximum extent possible, we will make use of previous information, reviewing results of previous evaluations of the data collection instruments and methodologies before we attempt to revise them. However, it is expected that this information will not be sufficient to refine our data collection instruments and methodologies without conducting additional research.
5. Minimizing Burden
The research to be requested under a generic clearance will be designed as relatively small-scale data collection efforts. This will minimize the amount of burden required to improve questionnaires, data collection instruments, and data collection procedures; to test new ideas; and refine or improve upon positive or unclear results from other tests. The results of the research conducted under this clearance are expected to improve the methods and instruments utilized in full scale studies and thereby improve information quality while minimizing burden to respondents.
6. Consequences of Less Frequent Collection
The research to be requested under a generic clearance will involve questionnaire and other data collection research/development activities. If the project were not carried out, the quality of the data collected would suffer. In addition, activities covered under the generic clearance will allow for more specific and precise calculation of burden hours and costs associated with survey and other data collection efforts conducted by OJJDP.
7. Special Circumstances
All the guidelines listed in the OMB guidelines are met. There are no special circumstances.
8. Consultations Outside the Agency
The 60-day Federal Register notice was published on April 18, 2017 (82 FR, No. 73, p. 18312). The 30-day Federal Register notice was published on June XX, 2017 (XX FR, No. XX, p. XXXXX). No public comments have been received in response to this notice.
Consultation with staff from other Federal agencies that sponsor surveys conducted by OJJDP will occur in conjunction with the testing program for the individual survey. Consultation with staff from other Federal laboratory facilities may also occur as part of joint research efforts. These consultations will include discussions concerning potential response problems, clarity of questions and instructions, and other aspects of respondent burden. Additional efforts to consult with potential respondents to obtain their views on the availability of data, clarity of instructions, etc., may be undertaken as part of the testing that is conducted under this clearance.
9. Paying Respondents
While no currently proposed projects involve the use of incentives, OJJDP may develop other projects where incentives could be used. OJJDP may offer up to $40 for any cognitive labs and up to $75 for focus group participation. We also may propose incentive experiments in limited cases.
10. Assurance of Confidentiality
According to 42 U.S.C. Section 3735, the information gathered under this clearance shall be used only for statistical or research purposes, and shall be gathered in a manner that precludes their use for law enforcement or any purpose relating to a particular individual other than statistical or research purposes. All respondents who participate in research under this clearance will be informed that the information they provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form and that their participation is voluntary. For personal visit and telephone interviews, this information will be conveyed verbally by the interviewer. For personal visit interviews, respondents will also be notified in writing to give them something they can keep and read. For self-administered questionnaires, the information will be included in the mailing package, recruitment communications and materials, either on the questionnaire, or the instructions. For Internet-based data collections, this information will be displayed prominently, and in a format that allows the respondent to print it out. All participants in cognitive research will be required to sign written notification concerning the voluntary and confidential nature of their participation. We will also inform respondents in writing of the need to have an OMB number. No participant direct identifiers will be maintained. The Generic Information Clearance will specify the particular authority for the data collection.
11. Justification for Sensitive Questions
It is possible that some potentially sensitive questions may be included in questionnaires that are tested under this clearance. One of the purposes of the testing is to identify such questions, determine sources of sensitivity, and address concerns related to those questions, insofar as possible, before the actual survey is administered. Justification for any sensitive questions included in a project covered by this generic information clearance will be included in the individual project submissions.
12. Estimate of Hour Burden
We estimate that the number of people involved in our exploratory, pilot, cognitive, and focus group work is estimated to cover approximately 2,000 respondents over the three year period, with a total estimated respondent burden of approximately 4,000 hours for the period from October 2017 through October 2020.
A variety of forms will be used in conducting the research under this clearance, and the exact number of different forms, length of each form, and number of subjects/respondents per form are not thoroughly known at this time. However, we can project that our activities will likely include testing items and data collection modes, cognitive labs or interviews, exploratory interviews, re-interviews, behavior coding and focus groups.
13. Estimate of Cost Burden
There is typically no cost to respondents for participating in the research being conducted under this clearance, except for their time to complete the questionnaire or participate in an interview or focus group.
14. Cost to Federal Government
It is very difficult to anticipate the actual number of participants, length of interview, and/or mode of data collection for the work to be conducted under this clearance over the entire 3-year clearance period. Without that information, it is not possible to estimate in advance the cost of the work under this Information Clearance to the Federal Government. Costs associated with each individual project will be covered by the statistical unit conducting the research and will come from their data collection budgets. We will include information about costs in the individual submissions.
15. Reason for Change in Burden
There is no change in burden, as this is an initial request for a generic information clearance.
16. Project Schedule
The research to be requested under a generic clearance is for the testing of various data collection methodologies and survey/questionnaire development. Data tabulations will be used to evaluate the results of the testing. The information collected in these efforts will not be the subject of estimates or other statistics in OJJDP reports; however, it may be published in research and development reports or be included as a methodological appendix or footnote in a report containing data from a larger data collection effort. The results of this research may be prepared for presentation at professional meetings or publication in professional journals. Due to the nature of this clearance, there is no definite or tentative time schedule at this point. We expect work to be conducted more or less continuously throughout the duration of the clearance.
17. Request to Not Display Expiration Date
No exemption is requested.
18. Exceptions to the Certification
There are no exceptions to the certification.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Hi All, |
| Author | Edith.McArthur |
| File Modified | 0000-00-00 |
| File Created | 2021-01-22 |