PRAMS Questionnaire Development Process

Attachment 3 PRAMS Questionnaire Development Process

Purpose

The PRAMS questionnaire is revised every 3 to 5 years. The goals of each revision process are similar. These goals are to maintain or reduce the length of the core survey; to ensure any emerging topics are captured; and to obtain internal and external stakeholder and partner feedback on the survey questions to ensure continued relevance and utility. The steps in the process, including evaluation of the previous phase questions, cognitive & field testing of new questions, and engagement of states and other stakeholders in the revision process are outlined below and remain constant across revisions.

Enhanced Workflow Description

1. The first step of the PRAMS questionnaire development process is initiated by the CDC PRAMS team. The CDC PRAMS team distributes a New Question Application for widely through state and federal colleagues and partners. This application should include the source of the question(s), performance (if known—e.g., evidence of validity, reliability, cognitive testing), public health importance, and analytic plan. As Phase 8 is the first time PRAMS has used a formal solicitation process that required funding, new questions or revisions to existing questions on topics that were not attached to funding were also considered for inclusion in the process described below.

2. For funded questions, the completed application is then submitted to the CDC PRAMS Team. A subgroup of the PRAMS team, the Questionnaire Work Group, is responsible for progressing questions through the questionnaire development process. It will act as a liaison between stakeholders and the PRAMS team for process adherence. The Questionnaire Work Group will receive applications and, if necessary, recommend to the applicant any preliminary changes.

3. The Questionnaire Work Group convenes and evaluates each application using predetermined evaluation criteria that measures whether or not new question applications adhere to PRAMS purpose. This group recommends (or denies) the application’s advancement to the next step in the process based on application’s adherence to the evaluation criteria.

4. Next, the Questionnaire Work Group presents the proposals to the PRAMS states. For the Phase 8 revision, there were not sufficient applications to necessitate a vote. All applications that met the evaluation criteria were accepted. However, following a comment period, states partners determined that several of the proposed core questions were better suited to be standard questions than core questions. The applicants were notified. In future revisions, a voting process will be used in the event that core question applications exceed available space.

5. Accepted questions from the funded applications, as well as any other unfunded new or revised questions are formatted for mail and telephone survey administration and translated into Spanish. English and Spanish mail and telephone versions are sent for cognitive testing. The cognitive testing includes in-depth private interviews with individual women. The purpose of cognitive testing is to identify question delivery issues that may come up during an actual interview.

6. Based upon the results of cognitive testing, changes may be recommended to the applicant. Mutually agreed upon changes are made to problematic questions, which then undergo 1 or 2 rounds of field testing with patients at clinics or doctors’ offices.

7. New questions are integrated into the core survey and presented to CDC subject matter experts and the PRAMS State Working Group, which includes volunteer representatives from 7 regional groups of states. This happens during 1-2 scheduled meetings with the CDC subject matter experts, and quarterly conference calls with the State Working Group. The information was also sent via email for comment to the PRAMS Advisory Group, a group of internal and external stakeholders.

8. Once finalized, the questions are added to the CDC PRAMS Model Protocol and sent to IRB for review. Beginning in 2015 the questionnaire will also be submitted for OMB review and approval.

9. Questions are sent to PRAMS states, which select their standard questions and develop any state-specific questions based on state priorities and input from their state steering committees that may consist of internal and external partners such as those from academia, non-profits, health care system, medical associations, etc. PRAMS states submit their question selection, and the PRAMS Questionnaire Work Group formats the mail survey files, creates the hard copy phone files, and collaborates with a software development contractor to complete PIDS (data entry & CATI) programming.

10. The survey is then administered at the state-level.