ASPEP Form Layout Testing Protocol
Draft 3/12/2012
Research Questions to Address:
Do the changes in layout adversely affect the response process, and the responses provided?
Are there any items that cause the respondents problems?
Explain purpose of meeting: to understand the process used to answer questions, to get feedback on proposed changes
Meeting with entities in different states, different types and of different sizes. Not everybody keeps their records the same way, so some questions will seem odd and/or obvious.
Explain that we are not testing the respondent – we only want information
Structure of meeting: general and specific questions about the form itself, the reporting organization and also how R reported data
Permission to record discussion? Have R sign consent form.
Before the Questionnaire:
What types of services does this government entity provide?
What is your role in this government entity? What kind of responsibilities do you have?
What was your role in the process for responding to this questionnaire? (Gathered data? Compiled data? Checked data? Authorized release?)
Reflect back R’s specific answer: “you said…”
In your own words, what is this question asking?
How did you arrive at this number/answer this question? / Tell me how you arrived at these answers. What did you include in this number? What did you specifically exclude?
What records (if any) did you look at? What line or lines were of interest?
Were any other people involved in the process of coming up with this number? [If Yes] What departments are they in?
How well does this data request match with your records?
How did you arrive at your answer?
Warm-up probes:
What is this question asking you?
How did you arrive at your answers?
What does the term “full time employees” mean to you?
What is the bullet under the word “exclude” asking you to do?
Pay careful attention to how the respondents grapple with the different layouts between sections.
General Probes for this section:
How would you go about completing this question?
What is this question asking you?
How did you arrive at your answers?
What information are you putting in each column?
What do you think about the payroll frequency codes listed on the page? Are they easy or difficult to find?
If applicable: How would/did you record full time and part time employees? (Make sure the respondent notices that full and part time payroll are concatenated down the page instead of across).
What do you think of the way these questions are laid out on the pages?
Is it easy or difficult to figure out how to report the information?
Which layout do you like best? Least? What did you like about the layouts? What didn’t you like?
Wrap-up Questions
What do you like about the current form that you don’t like about this new form we showed you today?
What do you like about the new form we showed you today that you don’t like about the current form?
About how long would it take you to complete this form (estimate in hours)?
Thank you for your help!
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | pick0002 |
| File Modified | 0000-00-00 |
| File Created | 2021-02-02 |