2015 FDIC Survey of Small Business Lending
Cognitive Testing Protocol
Ken Pick
Kenny Herrell
Research Questions:
Do respondents interpret the questions and instructions in a manner that meets the questionnaire’s intent?
Do respondents comprehend the terminology used throughout the questionnaire? (If not, what terminology are they using?)
Will respondents need to go to other people or departments within their organization to report the data requested on the questionnaire?
Overall, is the requested information available in respondents records?
Expected Length of Interview: 60 minutes (Note, in the interest of time and respondent burden, not all questions on the form will be asked about.)
Materials Needed:
2-3 copies of the draft questionnaire
Consent forms
Digital recorder
Business cards
Pen and/or pencils
Introduction:
People around the table: how long at bank, position/title and responsibilities
Explain purpose of meeting: to understand the process used to answer questions
Explain that we are not testing the respondent – we only want information
Structure of meeting: general and specific questions about the form itself, as well as the organization
Permission to record discussion? Have R sign consent form.
General probes to use throughout the interview:
Reflect back R’s specific answer: “you said…”
How did you arrive at this number/answer this question? Can you tell me more about this?
What records (if any) would you look at? What line or line items would be of interest?
Specifically, what would you include in this number? What would you exclude?
Before discussing the questionnaire:
To begin, can you please tell me a little about your bank?
(If needed) Have you filled out government forms in the past? Which ones?
(If needed) How are government forms completed at your bank?
Now, I’m going to hand you a copy of the draft questionnaire (may want to mention that the survey will entirely be electronic, not paper, when launched next year). Feel free to write in the questionnaire and think aloud as to how you would complete these questions. After you complete a question, I’ll ask you follow-up questions about the process you took to formulate an answer.
To begin, let’s look at the question on the top of Page 1.
In your own words, what is this question asking you?
(If needed) How do you know this information?
Okay, let’s turn the page and look at Question 1.
In your own words, what is this question asking you?
(If needed) Would you need to look at any records? Talk to anyone in your company? Tell us about that…
What does the term “small business” mean to your bank?
(If needed) What characteristics are used by your bank to define a small business?
(If needed) Are there any other criteria that should be added to this list?
(If needed) Does your bank define a small bank consistently across all loan products offered?
(If bank doesn’t have any criteria to define a small business) How would you go about answering this question?
Next, let’s look at Question 2 on the next page.
How would you go about answering this question?
(If needed) What records would you need to consult?
(If needed) How easy or difficult would it be to access these records?
Let’s turn the page and look at the next set of questions. (Pause after they read transition)
Questions 3-8.
(For each question, as appropriate) How would you go about answering this question?
(If needed) Would you need to look at any records? Talk to anyone in your company? Tell us about that…
(For each question, as appropriate) Can you describe the process you need to go through to access your bank’s CALL reports?
(if necessary) How easy or difficult is it to access the CALL reports?
(For each question, as appropriate) Can you describe how you would find the information you need in the CALL reports?
(If necessary) How easy or difficult is it to find the information you would need to answer the questions?
(If R says Yes to each question) Is this information tracked based on counts of loans, dollar amounts of loans, or both?
(If R says No to a question) How would you go about estimating this number? How confident would you be in your response?
(For each question) Is it easy or difficult to answer this question?
(Question 4) What does the phrase “Home Equity Lines of Credit” mean to your bank?
(Question 4) What line item would you pull this from in your bank’s CALL report?
Question 9.
How would you go about answering this question?
(If needed) Would you need to look at any records? Talk to anyone in your bank? Tell us about that…
What does the phrase “loan-to-one-borrower” mean to your bank?
Let’s turn the page. Take a moment to read the introduction at the top of the page and then let’s look at Question 10.
In your own words, what do the instructions before Question 10 tell you?
(Repeat general probes and follow-up probes for Questions 10-17.)
How would you go about answering this question?
(If needed) Would you need to look at any records? Talk to anyone in your bank? Tell us about that…
(If R says Yes) Is this information tracked based on counts of loans, dollar amounts of loans, or both?
(If R says No) How would you go about estimating this number? How confident would you be in your response?
Next, let’s turn the page and look at this next section. Question 18.
How would you go about answering this question?
What does the phrase “small business loan products” mean to you?
Is this question easy or difficult to answer? (If difficult) Tell us why this question is difficult…
Questions 19, 19A-19H.
In general, what are these questions asking you? How would you answer this question/report this number?
Looking at the sub-questions under Question 19, which ones are easy to answer? Likewise, which ones are difficult to answer? Why are these difficult to answer?
Does your bank use any other characteristics to define a loan product?
(If yes) Tell us about these other characteristics…
Questions 20/21.
How would you define your bank’s second most popular small business loan product?
Is it easy or difficult to define your bank’s second most popular small business loan product?
What about your bank’s third most popular small business loan product?
Is it easy or difficult to define your bank’s third most popular small business loan product?
Next, let’s turn the page and look at Question 22 and its sub-questions. Take a few minutes to complete them and then I’ll have some questions for you.
(For each question, as appropriate) How would you answer this question/report this number?
(If needed) Would you need to look at any records? Talk to anyone in your bank? Tell us about that…
Looking at the sub-questions under Question 22, which ones are easy to answer? Likewise, which ones are difficult to answer? Why are these difficult to answer?
(Question 22AA) Looking at Question 22AA, are there other regularly used reasons for exemptions that your bank uses but you don’t see listed? (If yes) Tell us about these…
Let’s turn the page and look at Question 25.
How would you answer this question for your bank?
What does the term geographic market mean to you?
In your own words, how would you describe to someone the geographic market your bank serves?
What about the term metropolitan statistical area – what does that mean to you?
In your opinion, are we missing any market configurations from the list provided?
Question 26.
How would you go about answering this question?
In your opinion, are there other institutions that are competitors but not currently listed?
How easy or difficult is it to rank your competitors?
How would you know if any of these new institutions are new competitors to your bank?
Question 27.
How would you go about answering this question?
How easy or difficult is it to rank the importance of these competitors to your bank? What criteria would you use to rank the importance?
Questions 28-31.
How would you go about answering this question?
Are there other areas that aren’t listed where your main competitor competes against you? If yes, what are they?
Question 31A.
How would you go about answering this question?
How easy or difficult is it to rank the importance of these forms of competition to your bank? What criteria would you use to rank the importance?
Question 32.
How did you arrive at an answer for this question?
Are there any of these options where you do not know whether a borrower can apply for a loan?
Question 33.
How did you arrive at an answer for this question?
How confident are you in your response?
Question 33A.
How did you arrive at an answer for this question?
How confident are you in your response?
Can you think of other types of applicants that are allowed to submit applications online not currently listed as a response option?
Question 34.
How did you arrive at an answer for this question?
How confident are you in your response?
Question 34A.
How did you arrive at an answer for this question?
Questions 35/35A.
How did you arrive at an answer for this question?
How confident are you in your response?
Are there other ways, not currently listed, that your bank uses to monitor small business loans? If yes, tell us about them…
Questions 36/36A/36B:
In your own words, what is this question asking?
What does the term “capital constraints” in 36A mean to you?
Is it easy or difficult to answer this question?
Questions 37/37S/38:
In your own words, what is this question asking?
Questions 39/40:
How would you answer this question?
In your own words, what is a general purpose reloadable (GPR) prepaid card? What does it include? What does it exclude?
(If needed) What is the term your bank uses for prepaid cards that bear the bank’s name upfront?
Questions 41A-E:
(For each question, as appropriate) In your own words, what is this question asking?
(If R selects multiple qualifying activities for Question 41B) Tell us more about this.
(If needed) Does this mean the customer engaged in any of these activities would have the fee waived or all of these activities would have the fee waived?
(Question 41E) How did you arrive at this answer? How confident are you in this answer? (If needed) Does your bank track this information?
Questions 42A-D:
(For each question, as appropriate) In your own words, what is this question asking?
(If R selects multiple qualifying activities for Question 42B) Tell us more about this. (If needed) Does this mean the customer engaged in any of these activities would have the fee waived or all of these activities would have the fee waived?
(Question 41D) How did you arrive at this answer? How confident are you in this answer? (If needed) Does your bank track this information?
Questions 43A-C:
(For each question, as appropriate) In your own words, what is this question asking?
(If R selects multiple qualifying activities for Question 42B) Tell us more about this.
(If needed) Does this mean the customer engaged in any of these activities would have the fee waived or all of these activities would have the fee waived?
Question 44:
How did you arrive at an answer for this question?
Question 45:
How did you arrive at an answer for this question?
What does term “automated overdraft” mean to you?
Question 46:
How did you arrive at an answer for this question?
How confident are you in this answer?
(If needed) Does your bank track this information?
Questions 47-56:
(For each question, as appropriate) In your own words, what is this question asking?
(For each question, as needed) Is this information available in your records? Would you need to talk to someone else to answer this question?
(Question 53) Does your bank have a loss threshold below which an applicant would not be able to open an account?
(Questions 54-56) Are there any differences in the eligibility requirements for a basic checking account, a basic savings account, and a network branded GRP prepaid card offered and issued by the bank?
(Questions 54-56) Are there any negative credit history triggers that your bank looks for to determine eligibility for opening a basic consumer account?
Wrapping Up:
Do you think you could complete this survey by yourself? Or would you need help from someone else? If the latter, who? And which department are they from?
How long do you think it would take your bank to complete this survey? Include the time spent by anyone who worked on it.
Overall, do you have any suggestions for us on how we can improve this survey?
Thank you for your time today!
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Kenneth M Pick |
| File Modified | 0000-00-00 |
| File Created | 2021-01-28 |