Design Overview

September 9, 2011
To Whom It May Concern,
Thank you for the feedback and guidance on our needs assessment survey to guide the manual
development. Based on our discussion and the OMB guidance document on survey design, we
have made significant changes to the supporting documents and survey such reducing the
number of open ended items, rearranging response sets, and counter balancing categorical
responses to reduce order effects. In addition to providing the two paragraph overview on webbased survey design, we have also programmed the survey in Survey Monkey for your review
(https://www.surveymonkey.com/s/SACollaborationAssessment). As discussed, since the
surveys across professions (law enforcement, prevention, education) are similar, we modified
and programmed one survey for law enforcement in Survey Monkey for your review. This will
allow us to have a master version to include all modifications. Once the survey is approved, we
will apply all your final recommendations to the other two versions of the survey.
Survey Design Overview
We will use the latest methods for survey administration and design to develop the provider
web-based surveys (Dillman, 2007; Groves, Fowler, Couper et al, 2009). Whether a respondent
will complete a long questionnaire is based on the level of responsibility, commitment and
interests so keep important and crucial content early. Establishing trust with your participants is
most important. Studies highlight that there are several important design considerations to
take into account when developing surveys in general and in particular web surveys (Dillman,
2000; 2007;Tourangeau, Couper, and Conrad 2004, 2007). Some important general
considerations include: a simple short descriptive title with all the sponsor information, make
sure directions are included as a part of each section embedded in the section, make sure initial
items are more general then get more specific, group similar questions to reduce cognitive
burden, use design to group items, and pay close attention to page breaks, make the flow of
the survey logical, using dark print for questions and light for response, blue and yellow
backgrounds improve survey response rates, placing answer choices in similar places.
While the above apply to nearly all surveys, there are particular issues with web-based survey
design. Studies highlight that the layout of the questions (Christian and Dillman 2004; Toepoel,
Das, and Van Soest 2006), the grouping of related items on a single screen is likely to lead
respondents to view the items as related entities, thus increasing the correlation among them
(Tourangeau, Couper, and Conrad 2004, 2007; Dillman 2007). Significant variation can occur
with web survey because of different formatting options as compared to web-based surveys
and it is important to understand how these may impact outcomes. LozarManfreda, Batagelj,
and Vehovar (2002) found that a one-page design results in higher item nonresponse than
different pages and that there are optimal times to group items on separate pages. For
example, Toepoel, Das, and van Soest (2009) participants react negatively to too many items on
a screen and Couper et al., (2001) highlights using single and multiple item screens to capture

data. The layout of the actual questions and responses are important as well. Traditional
surveys use horizontal responses but web-based surveys typically use vertical design. However,
this can be problematic as Toepoel, Das Van Soest (2008) recommend using a linear horizontal
layout without numbers for a five-point fully labeled rating scale when developing surveys
based on their research. Moreover, response sets should be counterbalanced. There is also
evidence that radio buttons are best at increasing response rates. While the research on
optimal methods to develop and deliver web-based surveys is growing, we have a fair empirical
base from which to develop the surveys for the current grant.