B. Collection of Information Employing Statistical Methods
1. Sample Design
The data collected will be used for research activities which improve data collection processes, rather than to produce estimates about the population. The objective is to interview a variety of people, rather than a probability sample of the population. For most of the research design activities concerning items applicable to the general population, respondents will be recruited by means of fliers or other advertisements posted in public places or in newspapers.
For testing some hypotheses, however, some initial screening of individuals will be done to identify eligible respondents. Eligible respondents are defined as those individuals who have not participated in more that 3 survey research projects in the preceding six months, or who meet other necessary requirements. Special attempts may be made to recruit from specific groups if there are no volunteers from these groups as a result of the general recruiting effort.
In addition, projects in the furtherance of Fed-State cooperative agreements or interagency initiatives may call for participation by state agencies, federal contractors, and other establishments. The cooperation of these organizations will be solicited through agency contacts and/or written correspondence to the appropriate department personnel.
2. Data Collection Procedures
Recruitment:
Potential respondents are typically solicited through newspaper advertisements that state briefly that individuals are needed to participate in research on surveys conducted by the Bureau of Labor Statistics, and that $40 compensation is offered. Persons responding to the advertisement are given a brief description of the nature of the research task. Those interested provide their name and a minimal set of demographic characteristics that are matched against the needs of the particular study. Eligible individuals are then scheduled for an appointment. Those not meeting current study requirements are placed in a respondent pool, and are eligible to participate in future studies.
Some projects require the use of a targeted sample, such as establishments involved in government sponsored surveys, reporting offices from state agencies, or organizations conducting contractual work for the federal government. Organizations will be asked to participate based on the requirements of the research design, and in accordance with the goals outlined above (A.2.). Prior to giving their consent they will be provided with: (1) a written description of the study, including details of study purpose, data collection methodology, and burden estimate; (2) a copy of the Privacy Act statement; and (3) a Consent statement explaining the use of the information collected and the voluntary nature of the study.
Telephone interview and mail-in survey studies that draw upon respondents from the general population use recruitment methods similar to those used for in-house laboratory sessions. However, individuals in these studies will receive additional written materials to include (as above): (1) a study description and estimate of interview/survey length; (2) the Privacy Act Statement; and (3) a Consent statement. Mail-in study participation and telephone interviews will be scheduled only after these materials have been read, and the individuals have given their informed consent, either verbally or in writing.
Lab Interviews:
Once a laboratory interview is scheduled, it is the responsibility of the respondent (from the public sector) to travel to the interview site. The BLS research rooms are located in Room 1950 on the first floor of the Postal Square Building, 2 Massachusetts Avenue, NE, Washington, DC. The rooms are private to insure confidentiality of the interview. To reduce the number of no shows, scheduled volunteers are phoned to remind them of their appointment.
When respondents arrive they receive an oral and/or written explanation of the purpose of the study and the research procedures. The respondent is then given a consent form to sign which includes a Privacy Act statement on the back. The consent form, in addition to the OMB number and expiration date, includes the OMB failure to comply notice, which states that if the OMB control number is not present, the respondent does not have to complete the survey. The need for audio or video taping of the interview is explained if such taping is planned and the respondent is asked to sign the consent form. If consent is not granted, the session will not be recorded. The study may last from 1/2 hour to 2 hours depending on the specific laboratory techniques applied.
The selection of the laboratory technique, in turn, is determined by the hypotheses to be tested. The most commonly used methods include concurrent and retrospective think-aloud interviews. In these interviews, respondents are asked questions (pertinent to the data collection instrument in question) and are asked to think-aloud about how and why they answered as they did. The interviewer usually probes extensively to ascertain the degree of comprehension and the recall processes involved.
Debriefing:
All respondents will be debriefed. This procedure explains the purpose of the project and answers respondents' questions regarding the study.
3. Methods to Maximize Response Rates
As noted, to reduce the number of no-shows, scheduled laboratory respondents will be sent a reminder letter giving the time of the interview and directions to the laboratory. They will also receive a reminder telephone call before the interview. Other data collection procedures will incorporate similar reminders to reduce the level of non-response.
4. Tests of Procedures and Methods
The tests proposed for research fall into a number of categories which cognitive psychologists utilize to confirm or reject research hypotheses. Some of these tests include those tasks outlined by Michael W. Eysenck (1984) in A Handbook of Cognitive Psychology. Some of the possible tests outlined are:
developing protocols, scenarios, and question probes--follow-up questions used to gain more information about respondents' strategies for answering questions,
concurrent think-aloud interview-- respondents think aloud while answering questions and responses are probed extensively,
focus groups and individual interviews-- Structured and unstructured discussion of the survey topic with groups or individuals,
retrospective think-aloud interview-- respondents answer all questions first, then are asked how they arrived at their answers,
sorting and ranking tasks--respondents sort lists or similar items into groups that go together and rank the items according to a specified scale,
confidence ratings--respondents relate the degree of confidence they have in the accuracy of their answers,
memory cues--interviewer reads terms which are intended as aids to recall,
response latency--measurement of the elapsed time between the presentation of the question and the respondent’s answer,
paraphrasing--respondents repeat the questions in their own words.
In addition, BSRL increasingly provides evaluation of and development assistance with BLS electronic data collection and data dissemination instruments (e.g., usability tests of Bureau websites, interviewing software, etc.). BSRL’s usability laboratory offers both on-site and remote testing capabilities.
5. General BSRG Procedures for Submitting packages to OMB
In accordance with study guidelines, studies originating from the BSRL submit supporting documentation that outlines the purpose, cost, and estimated burden. This documentation also provides a description of the study design, the data collection methodology, and the guidelines used for ensuring confidentiality, and includes copies of all relevant project materials (e.g., contact letters, collection instruments, confidentiality forms, protocols). A Nonsubstantive Change Request is submitted with each package. The request indicates the title of the study, the assigned project number (based on the fiscal year and number of submissions), and the number of burden hours requested. Within 10 working days OMB will review, provide comment, and take action on each package request. A written decision, including any terms of clearance are provided to BLS/DMS and forwarded to the primary investigator. At the end of each Fiscal Year, a summary report is submitted to BLS/DMS, including a statement of the actual burden hours used for each study. BLS/DMS then submits the report to OMB.
6. Statistical Consultants
The individual acting as a consultant to the Laboratory on statistical aspects of the basic research design is:
Dr. N. Clyde Tucker
Senior Survey Methodologist
Office of Survey Methods Research
Bureau of Labor Statistics
PSB Room 1950
2 Mass Ave., NE
Washington, DC 20212
(202) 691-7371
Individuals collecting data and analyzing information are:
Dr. Monica Dashen
Office of Survey Methods Research
(202) 691-7530
Dr. John Dixon
Office of Survey Methods Research
(202) 691-7516
Dr. Kathy Downey
Office of Survey Methods Research
(202) 691-7382
Dr. Jennifer Edgar
Office of Survey Methods Research
(202) 691-7528
Dr. Jean Fox
Office of Survey Methods Research
(202) 691-7370
Dr. Scott Fricker
Office of Survey Methods Research
(202) 691-7390
Dr. Brian Meekins
Office of Survey Methods Research
(202) 691-7594
Dr. William Mockovak
Office of Survey Methods Research
(202) 691-7414
Dr. Polly Phipps
Office of Survey Methods Research
(202) 691-7513
Dr. Christine Rho
Office of Survey Methods Research
(202) 691-7399
Dr. N. Clyde Tucker
Office of Survey Methods Research
(202) 691-7371
| File Type | application/msword |
| Author | Nora Kincaid |
| Last Modified By | Nora Kincaid |
| File Modified | 2008-09-24 |
| File Created | 2008-09-24 |