OMB Memo

Generic Information Collection Request

Cognitive Testing of National Survey of Psychiatrists Questionnaire

Request: The U.S. Census Bureau plans to conduct additional research under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). We propose to conduct cognitive interviews to pretest a proposed questionnaire for the National Survey of Psychiatrists (NSP). We are seeking approval for this project.

Background: The National Survey of Psychiatrists (NSP), is a new survey, sponsored by the Substance Abuse and Mental Health Services Administration (SAMHSA) and being developed by the Census Bureau. The central mission of the survey is to measure the inventory of available psychiatric care in the United States. The survey will be conducted by the Census Bureau using self-administered paper and web modes. The Census Bureau’s Demographic Statistical Methods Division (DSMD) Survey Methodology (SM) team was tasked with designing the survey instrument in preparation for the upcoming survey cycle. The SM team created a paper questionnaire based on technical requirements provided by SAMHSA regarding desired data to be collected. The next step is to conduct cognitive interviews to evaluate how well the questionnaire works in communicating the information we are intending to collect, and to identify any ways it may be improved.

Purpose: The purpose of this research is to interview recruited psychiatrists to assess their understanding of the questions and response options as presented in the proposed NSP paper questionnaire. In the cognitive interviews, we will assess the clarity and potential effectiveness of the survey questions and identify any pertinent respondent issues with comprehension or interpretation, with the goal of producing valid survey items that minimize burden and the potential for measurement error attributable to the questions themselves. The results of the cognitive interviews will influence decisions about final content and wording of the paper questionnaires for the first NSP survey cycle.

Population of Interest: The population of interest for the NSP is professionals currently licensed to practice medicine in the U.S., who have completed a residency in psychiatry. Participants will be screened during the study recruitment effort to ensure they meet criteria to test each major path of the NSP questionnaire, including: those currently providing direct clinical care as a psychiatrist, those not currently providing direct clinical care as a psychiatrist but who have within the past two years, and those who have never provided direct clinical care as a psychiatrist.

Language: The interviews will be conducted in English.

Timeline: Cognitive interviews will be conducted from December 27, 2018 to January 25, 2019.

Project scope: The cognitive testing of the proposed NSP questionnaire content will be carried out according to the schedule described in Table 1. In order for us to meet planned project deadlines for this work, we request OMB approval no later than December 21, 2018.

Table 1. Estimated Project Schedule

Recruiting (Ongoing)	Wednesday, 12/26/18 – Friday, 1/25/19
Conduct Cognitive Interviews	Thursday, 10/27/18 – Monday, 1/28/19
Data Analysis – Preliminary Results	Tueday,1/29/19 – Friday, 2/8/19
Develop Final Content Recommendations	Monday, 2/11/19 – Friday 2/22/19
Deliver Report	Monday, 2/25/19

General Protocol: Participants will be asked to think aloud while reviewing the proposed questionnaire. We will use concurrent and retrospective probing to get a full understanding of the participant’s thought processes and interpretations of specific terminology and perceptions of specific elements of the survey. After completing the task, each participant will be asked debriefing questions. All interviews will be audio-recorded to facilitate a summary of the results. Participants will be asked to sign consent forms and for permission to be recorded prior to the start of the cognitive interview. Due to the short duration of the interviewing period there will not be time to conduct two separate rounds of interviews. Therefore, the sampling and interviewing plan employs an iterative approach, where researchers will make revisions to question structure and wording to address any problems discovered during early interviews, testing these enhancements during later interviews.

Below is a list of materials to be used in this research.

1. NSP Questionnaire_Cognitive Interviewing Version (NSP-1_111418_D20)

2. NSP Alternative Cover page design (NSP-1_110618_v2)

3. Candidate NSP Cover Images (cover2 and cover3)

4. NSP Cognitive Interviewing Protocol

A copy of the cognitive interview protocol and the draft NSP paper questionnaires and cover page images are included in this request.

Sample: Up to a total of 30 participants will be interviewed. Participants will be recruited from the Washington, DC, Baltimore, MD and Phoenix, AZ metropolitan areas. Participants may be recruited and interviewed on-site during a conference or professional meeting of psychiatrists, if such an opportunity exists during the planned interviewing period. In such a case, participants may be recruited from additional geographic areas and interviewed on-site at a meeting or conference. Participants will be screened during the study recruitment to ensure at least 5 participants meet the criteria to test each of the three major paths of the survey so they are presentative of the NSP target population.

Recruitment: We will use a multi-pronged strategy to recruit participants, such as through personal networks, contacting professional organizations, advertising through social media, and recruitment at applicable conferences and meetings of professionals within the field of psychiatry. While not nationally representative, an attempt will be made to recruit participants with varying demographic characteristics (e.g., sex, age, race and ethnicity).

Use of Incentive: For this research, we request an increase in the typical incentive. The Census Bureau typically uses an incentive of $40 per one-hour interview. Because the NSP target population (psychiatrists) is highly educated and therefore may be hard to recruit for a research study, we propose offering up to $100 for their participation. There is a precedence for researchers (at the National Center for Health Statistics and U.S. Census Bureau) giving a $100 incentive for cognitive interviews of medical professionals.

Length of interview: Based on past experience with similar pretesting efforts it requires four screener conversations to recruit one participant. Each screener conversation lasts approximately three minutes. We estimate it will take 6 hours to screen and recruit 30 participants. We estimate it will take 60 minutes per participant to complete the cognitive interview. For 30 participants, the estimated burden for the interviews is therefore 30 hours, bringing the total burden to no more than 36 hours.

The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:

M. Christopher Stringer

Demographic Statistical Methods Division

U.S. Census Bureau

Washington, D.C. 20233

[email protected]

2