B. Collection of Information Employing Statistical Methods
1. Sample Design
The data collected will be used for research activities which improve data collection processes, rather than to produce estimates about the population. The objective is to interview a variety of people and establishments, rather than a probability sample of the population. For most of the research design activities concerning items applicable to the general population, respondents respondents will be recruited by means of fliers or other advertisements posted in public places or in newspapers. Additionally, online recruiting will be done using existing panels of volunteer respondents.
For testing some hypotheses, however, some initial screening of individuals will be done to identify eligible respondents. Eligible respondents are defined as those individuals who have not participated in more that 3 survey research projects in the preceding six months, or who meet other necessary requirements. Special attempts may be made to recruit from specific groups if there are no volunteers from these groups as a result of the general recruiting effort.
In addition, projects in the furtherance of Fed-State cooperative agreements or interagency initiatives may call for participation by state agencies, federal contractors, and other establishments. The cooperation of these organizations will be solicited through agency contacts and/or written correspondence to the appropriate department personnel.
2. Data Collection Procedures
Recruitment:
Potential respondents are typically solicited through newspaper or online advertisements that state briefly that individuals are needed to participate in research on surveys conducted by the Bureau of Labor Statistics, and that $40 compensation is offered. Persons responding to the advertisement are given a brief description of the nature of the research task. Those interested provide their name and a minimal set of demographic characteristics that are matched against the needs of the particular study. Eligible individuals are then scheduled for an appointment. Those not meeting current study requirements are placed in a respondent pool, and are eligible to participate in future studies.
Some projects require the use of a targeted sample, such as establishments involved in government sponsored surveys, reporting offices from state agencies, or organizations conducting contractual work for the federal government. Organizations will be asked to participate based on the requirements of the research design, and in accordance with the goals outlined above (A.2.). Prior to giving their consent they will be provided with: (1) a written description of the study, including details of study purpose, data collection methodology, and burden estimate; (2) a copy of the Privacy Act Statement; and (3) a Consent statement explaining the use of the information collected and the voluntary nature of the study.
Additionally, for some studies, online recruiting is done using existing panels of volunteer respondents. When using these panels, an advertisement describing the study is posted to the panel, and volunteers are able to review it and decide whether to participate or not. Those that agree to participate follow the screening and informed consent form process laid out below.
Telephone interview, online and mail-in survey studies that draw upon respondents from the general population use recruitment methods similar to those used for in-house laboratory sessions. However, individuals in these studies will receive additional written materials to include (as above): (1) a study description and estimate of interview/survey length; (2) the Privacy Act Statement; and (3) a Consent statement. Mail-in study participation and telephone interviews will be scheduled only after these materials have been read, and the individuals have given their informed consent, either verbally or in writing.
Lab Interviews:
Once a laboratory interview is scheduled, it is the responsibility of the respondent to travel to the interview site. The BLS research rooms are located in the Postal Square Building, 2 Massachusetts Avenue, NE, Washington, DC. To reduce the number of no shows, respondents are emailed a confirmation, and phoned a day or two prior to remind them of their appointment.
When respondents arrive they receive an oral and/or written explanation of the purpose of the study and the research procedures. The respondent is then given a consent form to sign which includes a Privacy Act Statement on the back. The consent form, in addition to the OMB number and expiration date, includes the OMB failure to comply notice, which states that if the OMB control number is not present, the respondent does not have to complete the survey. The need for audio or video taping of the interview is explained if such taping is planned and the respondent is asked to sign the consent form. If consent is not granted, the session will not be recorded. The study may last from 1/2 hour to 2 hours depending on the specific laboratory techniques applied.
Interviews and usability tests are conducted in the OSMR cognitive laboratory. Respondents complete the tasks in a private room, observerable only by OSMR researchers or program staff from the observation room. Respondents are informed during the consent process if there are observers.
The selection of the research technique, in turn, is determined by the hypotheses to be tested. For cognitive interviews and usability tests, the most commonly used methods include concurrent think alouds and retrospective probing. In these methods, respondents are asked questions (pertinent to the data collection instrument in question) and are asked to explain, either during their response process or afterwards, how and why they answered or behaved as they did. Other research techniques may include focus groups, card sorts, or vignettes.
Debriefing:
All respondents will be debriefed. This procedure explains the purpose of the project and answers respondents' questions regarding the study.
3. Methods to Maximize Response Rates
As noted, to reduce the number of no-shows, scheduled laboratory respondents will be sent a reminder email giving the time of the interview and directions to the laboratory. They will also receive a reminder telephone call before the interview.
4. Tests of Procedures and Methods
The tests proposed for research fall into a number of categories which cognitive psychologists utilize to confirm or reject research hypotheses. Some of these tests include those tasks outlined by Michael W. Eysenck (1984) in A Handbook of Cognitive Psychology. Some of the possible methods, which can be done in person or online, outlined are:
developing protocols, scenarios, and question probes -- follow-up questions used to gain more information about respondents' strategies for answering questions,
concurrent think-aloud interview -- respondents think aloud while answering questions and responses are probed extensively,
retrospective think-aloud interview -- respondents answer all questions first, then are asked how they arrived at their answers,
focus groups and individual interviews -- structured and unstructured discussion of the survey topic with groups or individuals,
sorting and ranking tasks -- respondents sort lists or similar items into groups that go together and rank the items according to a specified scale,
confidence ratings -- respondents relate the degree of confidence they have in the accuracy of their answers,
memory cues -- interviewer reads terms which are intended as aids to recall,
response latency -- measurement of the elapsed time between the presentation of the question and the respondent’s answer,
paraphrasing--respondents repeat the questions in their own words.
In addition, BSRC increasingly provides evaluation of and development assistance with BLS electronic data collection and data dissemination instruments (e.g., usability tests of Bureau websites, interviewing software, etc.). BSRC’s usability laboratory offers both on-site and remote testing capabilities.
General BSRC Procedures for Submitting packages to OMB
In accordance with study guidelines, studies originating from the BSRC submit supporting documentation that outlines the purpose, cost, and estimated burden. This documentation also provides a description of the study design, the data collection methodology, and the guidelines used for ensuring confidentiality, and includes copies of all relevant project materials (e.g., contact letters, collection instruments, confidentiality forms, protocols). A Generic Information Collection Request is submitted for each package. The request indicates the title of the study, the assigned project number (based on the fiscal year and number of submissions), and the number of burden hours requested. Within 10 working days OMB will review, provide comment, and take action on each package request. A written decision, including any terms of clearance are provided to BLS/DMS and forwarded to the primary investigator.
5. Statistical Consultants
The individual acting as a consultant to the Laboratory on statistical aspects of the basic research design is:
Dr. Jennifer Edgar
Director, Behavioral Science Research Center
Office of Survey Methods Research
Bureau of Labor Statistics
PSB Room 1950
2 Massachusetts Ave., NE
Washington, DC 20212
(202) 691-7528
Individuals collecting data and analyzing information are:
Dr. John Dixon
Office of Survey Methods Research
(202) 691-7516
Dr. Jennifer Edgar
Office of Survey Methods Research
(202) 691-7528
Dr. Jean Fox
Office of Survey Methods Research
(202) 691-7370
Dr. Scott Fricker
Office of Survey Methods Research
(202) 691-7390
Dr. Robin Kaplan
Office of Survey Methods Research
(202) 691-7383
Dr. Brandon Kopp
Office of Survey Methods Research
(202) 691-7514
Dr. William Mockovak
Office of Survey Methods Research
(202) 691-7414
Dr. Polly Phipps
Office of Survey Methods Research
(202) 691-7513
Dr. Erica Yu
Office of Survey Methods Research
(202) 691-7924
| File Type | application/msword |
| Author | Nora Kincaid |
| Last Modified By | SYSTEM |
| File Modified | 2018-03-20 |
| File Created | 2018-03-20 |