Form SBLS Protocol SBLS Protocol SBLS Protocol

Small Business Lending Survey (FDIC)

Centurion Usability Testing Protocol (2/17/2016)

(This protocol is a guide – the questions herein will not necessarily be asked exactly as worded or necessarily in this order. Please note that not all questions will be asked in every interview.)

Part A – Introduction

1. Introduce observers and their background

2. Purpose of visit:

Let me start by telling you a little about what we will be doing today. The Small Business Lending Survey is being conducted in order to provide data on banks’ lending habits to small businesses. Currently, some data is collected in the CALL reports, but much of the data in the CALL reports are not direct measures of small business lending, which is what we hope to accomplish with this survey.

This survey will only be available online. We are interested in getting feedback from respondents about the electronic version in order to identify needed improvements.

3. Permission to audiotape discussion? Have R sign consent form.

Before we get started, I would like to audio tape this interview, so I do not have to rely on my memory later. This session is confidential. Only persons connected with this project will have access to your tape. If that is all right with you, please sign this consent form. It also tells you about the confidentiality of this session.

Part B – Background Information

1. Respondent Background

• What is your role in this company/location?

• Have you completed any other government surveys? If so, which ones?

2. Electronic Reporting Background (if reported to other government surveys)

• Did you report electronically to any of those surveys?

• What made you choose to report electronically?

• What was your overall experience? Did anything stand out as being very helpful? Was there anything that you did not like?

Now I would like to show you an electronic draft of the SBLS form. Here is the website address for accessing the survey (show the respondent the URL, survey key (if applicable), test username and test password).

I would like you to go through the electronic version of the survey and complete it as you normally would. This is not a test of your ability or Internet savvy, so just proceed in whatever way makes the most sense to you. If you need to get information from records or others within the facility in order to answer a question, please just describe what you would need to do.

Do not hesitate to tell me anything you find surprising on the site, or if you have questions about the material on the site, just ask. We will try our best to answer your questions.

Part C – Testing

(Note: The primary interview technique is retrospective; R is asked about each question and/or group of questions after s/he has reported. Concurrent “think-aloud” probes are used when R indicates some confusion or other problem, e.g., by frowning, re-reading a question, looking away in thought, saying “Hmm”, etc.)

3. Login Screen

• Do you think that the information given on the Login Screen was sufficient?

• What are your overall reactions to the process for logging into the system?

• Is there any other information you would like to see on the screen when logging in?

4. Main Menu

• Can you describe to me what you see on this page?

• Where do you need to go in order to start completing your form?

• Was this clear? If not, what could be done to make the start of the form clearer?

• Is there anything else that you would need at this point in order to start completing the form? Anything else that should be included on this screen?

• Did you notice the toolbar on this screen? What do you think each of the options will do?

• Are there any other types of information that would be helpful to you?

5. Survey Item Screens

General Questions

• Do you have any recommendation for improving this screen?

Instructions – Note whether R appears to read instructions

• Do you recall whether you read any instructions on this screen?

• If you had questions about definitions or the survey items, where would you go?

• FOR QUESTIONS WITH INFOBOX: If you wanted more information on what [infobox item], how would you get that information?

Edit failure messages – If the R has not yet triggered an edit, have them do so

on purpose

• Do the messages on the screen clearly indicate to you that there is an issue, or are they not clear? Can you explain to me what the problem(s) are?

• Is there a better way that we could notify you about the problem(s) on this screen?

• What would you need to do if you wanted to ignore the problem message you were receiving and move on to the next questions? Is it clear to you what action you need to take?

Underlined text – address early on

• Why do you think the text is underlined here?

• Observe if respondent ever tries to click on underlined text as if it were a link

Navigation

• If you needed to return to a prior section, how would you do that?

• Did you have any problems navigating back to a prior screen/section? Do you have any recommendations for improving the navigation?

• What did you think about the layout of the items on this screen? Was it a comfortable amount of information? Is there anything we could do to improve this screen?

“Select all” instructions

Note whether Rs attend to the “select all that apply” instructions

• Would it ever be possible for more than one answer to be correct in this question? What would you do if that were the case?

Mark only one questions

• Note if respondents attempt to select more than one option

• Did you try to select more than one option on this question?

For Dollar questions

• What unit are you reporting this answer in?

• How did you come to that conclusion?

Matrix questions

• What relationship do the columns have with each other?

• What relationship do the rows have with each other?

Note if respondent attempts to enter data in Autocalculate field

• If R tries to enter data in autocalculate: I noticed you trying to enter data in that field. Can you tell me what you were going to enter?

Capitalized text

• Did you notice that some words were printed in capital letters in the question on this page? What does that mean to you?

“Select only one” instructions

• Note whether Rs attend to the “select only one” instructions

• Do you recall seeing the instruction to select only one answer?

• (If R attempted to select more than one answer) Can you tell me about how you decided to select more than one answer in this question?

6. Review your responses

• What are your first thoughts when seeing this screen?

• Can you explain to me what is expected of you? What is this screen telling you?

• (If there are edits) How will you go about fixing this/these problem(s)? When you have finished fixing the problems, what do you need to do next? How will you submit the form?

• What would you do if you were unable to resolve an error?

• What do you think of the title of this screen? Is it a logical title?

• What other types of review would you be interested in performing prior to submission?

• Does having the option to review a PDF of your information at this time make sense?

• If you weren’t ready to submit your data at this point, what would you do?

7. Completion Certificate

• Is there any other information that you would need to have for your completion?

• Would you save a copy of this information for your records?

• If yes: how would you do that?

8. Main Menu (after submission)

• Would you save a copy of the data for your records?

• Where would you need to go on this site in order to do that?

Part D – Wrap Up

• Would you complete this on paper before completing online?

• Do you think that you would complete the online form in one sitting?

• If you were completing this form electronically, would you still want/need the paper form? If so, what purpose would the paper form serve?

• Now that you have gotten a good feel for the online form, what is your overall impression of it? Was it easy, hard?

• Did you have any issues with the navigation on the site?

• What are some of the things that you liked the most about the online form?

• What are some of the things that you liked the least about the online form?

• Are there any other options that we should provide in the online form?

• Do you have any suggestions for how to make this online form easier for you to use?

Thanks for your time and input!

8

SBLS (Centurion) Usability Testing Draft Protocol

February 17, 2016