Screening/Participation Instruments

Qualitative Testing of Integrated Mortgage Loan Disclosure Forms

Round 5_Moderator Guide_Industry 101111 to CFPB.DOCX

Screening/Participation Instruments

OMB: 1505-0233

Document [zip]
Download: zip | pdf


OMB No. 1505-0233

Moderator’s Guide—Lender/Broker


Greet participant and thank him or her for coming. Ask participant to take a seat and if he or she had any trouble finding the site or if he or she has been there before.

Introduction of Moderator and Notetaker (10 minutes)

Welcome, and thank you for coming today. My name is _________________ and this is ________________. We are from the Kleimann Communication Group, a design and research firm in Washington, DC. Before we get started, I want to let you know that I will be reading from a script. We are talking with a number of people this week, and we want to be sure we say the same things in the same way to everyone.

The material we are testing today is part of a study about the disclosure you provide when a consumer applies for a mortgage loan to buy or refinance a home. The study is being conducted by the Consumer Financial Protection Bureau, a government agency, also known as the CFPB.

N ote to Moderator: Hand participant piece of paper with the CFPB website address:

www.consumerfinance.gov/knowbeforeyouowe/


Representatives from the CFPB are observing today. They are very interested in hearing your thoughts. Let me know at the end of the session if you would like to meet them.

We know that as a lender or broker you interact with consumers using these forms and undoubtedly have important suggestions. We also want to get feedback from you about improving the form’s usability, reducing burdens, and easing implementation for industry. The CFPB is working to create a disclosure that consumers, lenders, and brokers can all understand and use. What you say to us today will help us do that.

I will be leading today’s session and __________ will be taking notes to help us remember what you say. We will be audio- and/or video- taping this session, based on what you agreed to, to ensure that we collect complete information. The entire session will take no more than 90 minutes, and we will take a break about half way through the session. The questions we will be asking have been approved by the Office of Management and Budget and have been assigned OMB Control Number 1505-0233.

Okay? Any questions?

Confidentiality

I have a few questions to ask you about the paperwork you filled out when you arrived. Please answer either yes or no to each of the questions that I will ask you. (If participant nods or shakes head, ask him or her to say the answer out loud for our tape)

  • Did you fill out a questionnaire when you arrived today?

  • Were you given consent forms to participate in this session and for us to audiotape and videotape your session today?

  • Did you read and sign those forms?

  • Did you give permission for both audiotape and videotape? (If did not give permission for video, say, “One of my colleagues in back will make sure that the video is off. “)

NOTE TO MODERATOR: Do not proceed with the session unless the participant answers “yes” to each of the first three questions. Participant must agree to audio but not to video.

Do you have any questions?

All of the information we collect today will be kept confidential to the extent permitted by law, and we will not identify you by name when we compile our results. In addition, we will not use your full name, address, or any other personal identifying information other than your image in reports, papers, or videos based on this research. In addition, I want to remind you that you may end the interview at any time.



About the Session

Over the next 90 minutes, I am going to ask you to do a couple of things and to respond to a series of questions. As I go through these questions, and you give me your responses, please remember there are no right or wrong answers. We aren’t testing you. We’re testing the disclosures and any information you give us is good information.

Introduction to Think-Aloud Technique

One of the best ways to learn about a disclosure is by watching someone interact with it for the very first time. Once I give you the disclosures, I’m going to ask you to talk out loud about what you are thinking as you read them. You might be reading out loud and thinking, “Wow! This is great!” or “I don’t like this!,” say those things out loud. Or maybe you think “This doesn’t work.” Say it out loud. We want to hear that inner voice in your head that comments on things you read. Talking out loud is the only way we can hear what you are thinking and how you are interacting. This may seem a little odd, but we really want to hear everything that you are thinking as you work with this disclosure for the first time, so we can improve it.

N ote to Moderator: If needed



To learn from you, it is important that you:


  • Tell us out loud where you are in the disclosures.

  • Tell us what you are reading, skimming, skipping in the disclosures.

  • Tell us any questions, suggestions, or confusion that you might have about any part of the disclosure.

Talking out loud is very important because our notetaker will be recording what you say as you go through the disclosures for the first time. What you are thinking is more important than what you are reading.





I know this technique of thinking aloud might seem unusual. So, I’d like to let you practice. I’m going to give you a menu from a restaurant and I’d like you to “think aloud” and tell me how you would go through this menu and decide what you’d want to eat. Again, I want to hear what you are thinking, reading, skimming, skipping, how you react to the menu, and how you go about making a decision. Any questions?

Note to Moderator: Hand participant Menu.

Now, here is the menu. Remember, that you are reading to decide what to order. To get you started, what is the first thing you see or notice?

N ote to Moderator:



  • Give the participant the menu and get him or her comfortable with talking about what he or she is looking at or reading, and with voicing questions, confusion, and decisions.

  • Coach the participant to use the technique. Be sure that he or she has looked at and commented on the following:

  1. the appearance—color, font, layout

  2. symbols and graphic elements, such as pictures or symbols

  3. details, such as why some items have descriptions and others don’t

  4. order in which he or she reads the information

  • Ask the participant what he or she is reading, where he or she is looking, and when a decision can be made. The task should take no more than 5 minutes.

  • Important: This task sets the stage for how well the participant will “talk aloud” in Task One.


Great! You understand exactly what we are looking for. Do you have any questions before we get started?

Task One: Think Aloud-ARM Loan Comparison (35 minutes)

Okay, I’m going to give you a scenario.

Note to Moderator: Hand participant Scenario Card and read out loud.


Industry Scenario- Purchase

You are contacted by an individual who wants to purchase a home. The house costs $180,000 and she is planning to put 10% down ($18,000) so wants to borrow $162,000. She has already paid $10,000 of the $18,000 as a deposit to the seller.




Purchase price of home: $180,000
Down payment: $18,000 (10% down)
Amount she wants to borrow: $162,000




In a moment I am going to hand you two mortgage loan disclosures. In reviewing the mortgage loan disclosures, I’d like you to imagine that you are reviewing these disclosures in preparation to explain them to this individual who has not purchased a home before. Your task is to look at these disclosures to decide if the disclosures provide the right information for consumers to be able to understand and compare loan options.

When I hand you the disclosures, I want to hear what you are noticing and thinking about as you look through the disclosures as well as your thoughts, questions, and ideas about the mortgage disclosures. Tell me what you like about what you see, what you don’t like, and what you think might be confusing to consumers. Remember we want to hear your thoughts—both positive and negative, so we can improve the disclosures to make sure consumers have clear information about the loan terms and can use the disclosures to compare loans.

Please go through the disclosures the way you would if you were reviewing them to present to a consumer and trying to anticipate where you may have to explain things. As you go through them, please speak aloud so I can hear what you are thinking just as we did with the menu. Remember this is a test of the disclosures, not a test of you. Do you have any questions before we get started?

Okay, here are the disclosures.

Note to Moderator: Hand participant mortgage disclosures.

To help get you started, what is the first thing you notice?

N ote to Moderator:



  • Allow participant time to review the disclosures and comment on them.

  • Ask participant to put disclosures on table.

  • State aloud whenever the participant moves to another section or page.

  • Ask participant what he or she is looking at.

  • If participant looks confused, ask “is there something that is confusing there?”

  • Remind the participant to tell you what he or she is thinking, what he or she likes and dislikes, and what he or she understands and doesn’t understand.

When the participant seems to be finished, ask the participant:

  1. Do you think a consumer would have trouble understanding the overall disclosures? PROBE: Why? Why not?

  2. If a consumer were provided with these 2 options, on a scale from 1-5, with 1 being really easy and 5 being very difficult, how easy or difficult do you think it would be for your customer to understand the differences between the loans on their own? (ADD: we know consumers benefit from talking to their lenders or brokers but we want to know if you think the design is effective in helping consumers understand the basics on their own)

  3. Imagine that I’m your customer and that I brought you both of these to help me understand them. Would you take a minute or two and show me how you would explain these two loans to a consumer using these disclosures?

  4. Which of these two loans would you recommend to me? Why?

  5. On page 1, does the design help draw attention to the key differences between the two loans, to the information that consumers should see and know? What is the key information for consumers?

  6. On page 1, is the design presenting the different features of these products in an even-handed way? Is it directing consumers to or away from a particular product or feature?

  7. On page 1, do you think consumers will have any trouble understanding anything in Loan Terms? Do you have any suggestions for improvement?

  8. On page 1, do you think consumers will have any trouble understanding anything in Projected Payments? Do you have any suggestions for improvement?

  9. In the Principal and Interest line under Payment Calculation, do you think consumers will understand what the minimum and maximum refer to?

  10. What do you think about rounding numbers vs. showing an exact number?

  11. On page 1, do you think consumers will have any trouble understanding anything in Closing Costs? Do you have any suggestions for improvement?

  12. On page 2, is the design drawing attention to the information that consumers should see and know?

  13. On page 2, is the design presenting the different products in an even-handed way? Is it directing consumers to or away from a particular product or feature?

  14. What do you think about the general organization of the page (showing the settlement costs in column 1 and the cash to close in the other)?



  1. In the Origination Charges, section A, is this the appropriate level of detail? If not, what information should be added or removed? Why?


  1. What do you think about sections B and C? Do you think consumers will easily understand what they can shop or negotiate on and what they can’t?


  1. What do you think about section D? Do you think consumers will easily understand this section? Do any other specific taxes need to be listed? Why?

  2. In the E. Prepaids section

    1. Is this the right level of detail in Section E?

    2. Do you have any suggestions for making this easier to understand?



  1. In the F. Initial Escrow Payment at Closing section,

    1. How do you explain the information in this section to your customers?

    2. Is this the right level of detail in Section F?

    3. Do you have any suggestions for making this easier to understand?



  1. In the first column Settlement Cost Calculation section

    1. Is the information in the right order?

    2. Is there other information that you would want to include here?

    3. Is there information that doesn’t need to be included?




  1. In the second column Cash to Close Calculation section

    1. Is the information in the right order?

    2. Is there other information that you would want to include here?

    3. Is there information that doesn’t need to be included?


  1. Do you think consumers will have any problems understanding the Adjustable Interest Rate Table? Suggestions?


  1. Do you think consumers will be able to connect this information back to the information on page 1?



  1. If you were working with a customer who received this disclosure, what would you recommend she do if she wanted to lower her closing costs? Are there any individual closing costs that she can reduce? How?


  1. Do you think consumers will understand the tolerance or limits on increases language? Do you have any suggestions for how to make it clearer?

  2. How does this tolerance language and presentation compare to the current GFE?


  1. If someone wanted to take advantage of or mislead consumers using this form, how could they do this? Follow up: Any other ways?


  1. What (if anything) would you have to do differently when working with a consumer using this disclosure compared to the current GFE and Truth in Lending statement?

  2. Would this disclosure eliminate the need to give or develop a worksheet or some other explanatory materials to help your customers understand the loan? How could you adapt this disclosure to replace any worksheets you provide now?

  3. Are there enough blank lines on this form to fit your fees?

  4. There is currently a line for “other fees” in Section D of the disclosure, how will that affect how you show your fees?

  5. Would an “other fees” section work well in any of the other sections?

  6. Are there any fees that could not be included in the “other fees” line because state or local law requires that they be itemized?

  7. Would you have or want to use extra pages or an addendum or worksheet to show any fees?

  8. Do you have any additional suggestions to improve the integrated disclosure?



We are just about half way finished. Let’s go ahead and take a short break. I’ll take you back to the waiting room and you can get a drink or use the restroom, and I will come and get you in a few minutes. Thanks.



Break (5 minutes)






Task Two: Fixed Loan Comparison (25 minutes)

Thank you for working with those disclosures. Now, I’d like to show you two other disclosures. These are for different products.

I’d like for you to go ahead and familiarize yourself with these disclosures and the different products, and then I just want to ask you a few questions about them. Just let me know when you are done looking through them. If anything jumps out at you or you have additional suggestions, please let me know what you are thinking. Any questions?

Note to Moderator: Hand participant second set of disclosures for fixed products.

Give the participant time to read and then ask:

  1. Do you think the overall design of this disclosure works well with a fixed loan product? Why?

  2. Do you think a consumer would have trouble understanding the overall disclosures with these different products? PROBE: Why? Why not?

  3. On page 1, is the design drawing attention to the key differences between the loans, to the information you want consumers to see and know.

  4. On page 1, is the design presenting the two different loans in an even-handed way? Is it directing consumers to or away from a particular product?

  5. On this product, the last period has an exact total payment (because it does not include any rounded components), do you think this would be confusing to consumers? If yes, why?

  6. On page 2, is the design drawing attention to the information that consumers should see and know?

  7. On page 2, is the design presenting the different products an even-handed way? Is it directing consumers to or away from a particular product or feature?

  8. Look at the Adjustable Payment Information table, do you think consumers will have trouble understanding this table? Suggestions?

  9. Do you think consumers will have trouble with any of the language? In Neg Am, do you think using “N/A” will be confusing? What about the “limits on changes” section of the table, will that be confusing? Suggestions?

  10. Do you think consumers will have trouble connecting the table information to the information on page 1? Suggestions?



  1. Looking at all 4 of these loan offers, if a consumer were provided with these 4 options, on a scale from 1-5, with 1 being really easy and 5 being very difficult, how easy or difficult do you think it would be for your customer to understand the differences between the loans on their own to be able to select a loan that is best for them?




Task Three: Page 3 (15 minutes)

Note to Moderator: Place all 4 page 3’s in front of participants

We are almost done but have one final task. I want to ask you some questions about page 3 of the disclosure.


Here are all 4 of the page 3’s, we’ll go ahead and spread these out in front of you. .


When participant seems finished ask:



Let’s look at the top of Page 3, the Comparisons section.


  1. What is your overall impression of the top of Page 3?

  2. Looking at the Comparison section, can you explain to me in your own words how you think consumers would use or not use that section?

  3. What do you think the “In 5 Years” section is trying to show you?

  4. How could consumers use the “In 5 Years” information to compare loans?

  5. How does the APR relate to the interest rate of the loan?

  6. How could consumers use APR information to compare loans?

  7. What do you think the Total Interest Percentage (TIP) is trying to show?

  8. How could consumers use the TIP information to compare loans?

  9. What do you think Lender Cost of Funds is trying to show?

  10. How could consumers use the LCF to compare loans?

  11. How could consumers use the LCF incorrectly?

  12. Of these four items in the Comparison section, which ones would be most useful in helping consumers to compare across loan offers? Why?

  13. Of these four items in the Comparison section, which ones would be least useful in helping consumers to compare across loan offers? Why?

  14. Any other comments about that top part Comparison section?

Now let’s look at the bottom of Page 3.


  1. Looking at the “Other Considerations” section, will you please look at each item and tell me if you think consumers may have trouble understanding any of the items? Why?


  1. What does it mean if a consumer signs this form?


  1. Given that the signature only acknowledges receipt, how important do you think it is to include it on the disclosure? Why?


  1. What, if anything, would you have to do differently to implement this form into your practice? How easy or difficult would implementation be?


  1. On a scale of 1 to 5, with 1 being the worst design you’ve ever seen, and 5 being the best design you’ve ever seen, how would you rate this design?


  1. Just a few more questions. What do you think about the current title of the disclosure “Loan Estimate” given that this will replace both the current GFE and the TIL statement?


  1. Here are a few names listed on index cards for the disclosure that we have been playing around with. Could you look through these and sort them into two piles: Those you like or think would work vs. those you dislike or think would not work? (READ NUMBERS ALOUD FOR RECORDING)


  1. Do you have any other name suggestions?


  1. Do you have any other comments to make?

Closing Remarks (1 minute)

Thanks so much for sharing your thoughts and impressions with us today. Your feedback and suggestions have been very helpful. Do you have any questions I can try to answer? Are you interested in meeting any of the observers? Ok. Thanks again for coming in today, you can return to the front desk to pick up your payment.

12


File Typeapplication/zip
File Modified0000-00-00
File Created2021-01-31

© 2024 OMB.report | Privacy Policy