Addendum - 0072

Addendum - 0072 (Final).docx

Continuing Disability Review Report

Addendum - 0072

OMB: 0960-0072

⚠️ Notice: This form may be outdated. More recent filings and information on OMB 0960-0072 can be found here:

2023-05-01 - No material or nonsubstantive change to a currently approved collection

Document [docx]

Download: docx | pdf

Addendum to the Supporting Statement for Form SSA-454-BK

Continuing Disability Review Report

20 CFR 404.1589 & 416.989

OMB No. 0960-0072

Background 1

Revisions to the Information Collection 3

Terms of Clearance 14

Public Comments on the Information Collection 17

60-day Comment Period Federal Register Notice 17

Overall Comments on the Time Estimate and Length of the SSA-454-BK………………………17

Overall Comments on the Complexity of the CDR Process and Stress on the Respondents……….21

Comments on the Frequency of CDRs………………………………………………………….23

Comments on Improving Form SSA-454-BK…………………………………………………24

Comments Regarding Specific Revisions to Form SSA-454-BK………………………………26

Comments Suggesting SSA Complete a Behavioral Audit on the CDR Process and Form………35

Comments on the new i454 Internet Modality…………………………………………………37

30-Day Comment Period Federal Register Notice (FRN) 37

Usability Testing Notes from the Round 1 i454 Sessions Held from 10/18/21 – 10/27/21 37

Usability Testing with SSI and SSDI Recipients (10/18/21 – 10/27/21) 37

Usability Testing with Advocacy Group Members (10/25/21 – 10/27/22) 49

Round 1 Usability Testing Overall SSA Evaluation 67

Usability Testing Notes from the Round 2 i454 Sessions Held from 3/29/22 – 4/8/22 68 Usability Testing with Advocacy Group Members (3/29/22 – 4/1/22) 68

Usability Testing with SSI and SSDI Recipients (4/4/22 – 4/8/22) 92

Round 2 Usability Testing Overall SSA Evaluation 115

Background

Respondents use Form SSA-454-BK to complete a mandatory review for the continue disability review (CDR) when SSA identifies them as needing an updated assessment of their disability status. Currently, Title II or Title XVI disability recipients can complete the Continuing Disability Review Report using one of two modalities: 1) a paper application or fillable PDF (using Form SSA‑454-BK); and 2) a field office interview, during which SSA employees enter claimant’s data directly into the Electronic Disability Collection System (EDCS).

Given the high number of CDRs SSA conducts per year, we identified the need to create an Internet-based modality for the Continuing Disability Review Report. This allows the respondents to submit their SSA-454-BK information electronically, with no need for an in‑person interview with a claims representative. To meet that need, SSA created the new i454, an Internet-based application which respondents can use to submit the SSA-454-BK online through SSA’s website. We authenticate the respondents prior to their accessing the i454 screens, so we can allow them to submit the screens electronically using a Submit button (in lieu of an eSignature).

The i454 screens mirror our EDCS screens, in that they use dynamic pathing that allows respondents to complete only those questions they need to answer based on previous responses. In addition, the system includes both help screens and instructions to allow the respondent to navigate the screens and answer the questions without needing to contact a field office representative. Once the respondent completes the i454 screens, the system allows them to review their answers and then submit the information directly to SSA by clicking on the Submit button. When the respondent submits the i454, SSA’s system propagates the data we receive into EDCS for processing and storage.

SSA completed initial usability testing on these screens to assess how they will work for the respondents. We conducted limited usability testing on the new screens internally with stakeholders from ODP, OPSOS, ODD, OEST and the SSA Boston Regional Office. Our external public facing usability testing took place on August 16, 2021, and ran through August 20, 2021. For the external testing, we included beneficiaries from the Social Security Title II disability and Supplemental Security Income programs while canvasing a range of ages, as well as members from the advocacy community.

SSA reviewed the feedback received through the initial usability testing for the i454, as well as the public comments from the 60-day Comment Period, which began on August 18, 2021, and ended on October 18, 2021. Based on this feedback, we conducted an internal workgroup to address the comments and revised the paper SSA-454, and the corresponding i454, to streamline the form and reduce the overall burden on the public.

After revisions to the SSA-454 were complete, SSA conducted a second round of usability testing on the i454. Our external public facing usability testing took place on March 29, 2022, and ran through April 8, 2022. For the external testing, we included beneficiaries from the Social Security Title II disability and Supplemental Security Income programs while canvasing a range of ages, as well as members from the advocacy community. We used this feedback to refine the revised form. We will continue to monitor the i454 once we implement and will meet with both respondents and advocacy groups to discuss potential enhancements or issues with the system so we can improve it as needed.

SSA also collected comments from the public on the paper version of the SSA-454. Our external public facing usability testing took place on May 3, 2022, and ran through May 10, 2022. For the external review of the paper SSA-454, we included beneficiaries from the Social Security Title II disability and Supplemental Security Income programs while canvasing a range of ages, as well as members from the advocacy community. We used this feedback to refine the revised paper form.

We will initially implement the new i454 screens for individual adult claimants, or their representatives, to use for medical CDRs only. While the agency conducts non-medical reviews, called redeterminations, for Title XVI Supplemental Security Income (SSI) recipients, we are not using the i454 for this purpose at this time. Instead, we use Form SSA-8202-BK (OMB No. 0960‑0145) for SSI redeterminations. Once we assess how well the system works for these respondents, we will work on expanding the system for use with more CDR respondents.

SSA plans to implement the new i454 screens in December 2022. To that end, we ask OMB to approve this information collection request by October 31, 2022, to give SSA enough time to implement the updated screens.

Revisions to the Information Collection

SSA is making the following revisions to the paper SSA-454:

Change #1: We revised the language under the “Before Completing This Report” section. We modified the language to provide plain language and clear instructions on what is needed to complete this report. This revision also included the removal of gender specific language throughout the form. We revised the language revised to “The office that reviews your medical condition will use the information in this report to decide whether you are still disabled. Please complete as much of the report as you can.”

Justification #1: We received multiple public comments that suggested the form be written in plain language; therefore, we implemented changes to the paper SSA-454 based on public comments as it is currently difficult to read, and that it would also be helpful for the form to include pictures for those who are unable to read.

Change #2: We revised the language under the “IF YOU NEED HELP” section. We modified this language to provide clear instruction on how to contact SSA when you need help completing this report. We added the “1-800 number to reach SSA by phone” and we added that “If you cannot speak or understand English, we provide an interpreter for free of charge.” We removed the NOTE: If you are assisting someone else with this report, please answer the questions as if that person were completing the report.

Justification #2: A commenter suggested that SSA revise Pages 1-2 of the instructions to include a statement with contact information for how to request assistance in completing this form by contacting an agency interpreter for the benefit of a person with a disability or individual helping a person with a disability who cannot speak, read, write, or otherwise understand English. Therefore, we included language informing the respondent that we provide an interpreter free of charge and giving the respondent a number to call to request this service. This commenter further claimed to understand that SSA policy states that any notice sent to a Limited English Proficiency (LEP) individual requiring contact regarding a claim must include the following statement: “We provide free interpreter services to conduct your Social Security business. However, if you prefer to have your own interpreter, you may do so, but with the understanding that our own interpreter may be present.” Even so, the commenter noted that many of their clients have barriers to accessing benefits, in part, because they lack an understanding of English. While the commenter understands that SSA does have policies to govern the use of interpreters, they believe a lack of an interpreter to translate notices and other documents (such as the CDR form) will continue to be an unnecessary hurdle for individuals with Limited English Proficiency (LEP). We believe these revisions directly respond to that commenter. In addition, removing the NOTE in this section reduces redundancy and clarification of this information was added to the narrative under SECTION 1.

Change #3: We revised the language under the “WHAT YOU NEED TO COMPLETE THIS REPORT” section. We modified this language to provide clear instructions on what information the respondent may need to gather prior to starting the CDR form. We removed the following:

• Print or write clearly.

• Include a ZIP or postal code with each address.

• Provide complete phone numbers, including area code. If a phone number is outside the United States, provide International Direct Dialing (IDD) code and country code.

• If you cannot remember the names and addresses of your health care providers, you may be able to get that information from the telephone book, Internet, medical bills, prescriptions, or prescription medicine containers.

We added the following language:

Name, address, and phone number of a friend or relative (other than your doctors) we can contact who knows about your medical conditions, and can help you with your case, if needed.
Name, address, and phone number of your health care providers you have seen within the last 12 months. (You may be able to get that information from the telephone book, Internet, medical bills, prescriptions, or prescription medicine containers.)

Any prescription or non-prescription medicines you take or have taken in the last 12 months.

Name of organization who we can contact that would have medical information about your condition(s) in the last 12 months. (Such as social services agencies, welfare agencies, attorneys, prisons, workers’ compensation and insurance companies who have paid you disability benefits.)

Information about any education since your last disability decision. (See top of Page 3 for date of last decision.)

Information about any vocational rehabilitation, employment, or other support services since your last disability decision. (See top of Page 3 for date of last decision.)

Justification #3: One commenter suggested SSA add a list of “What You Need to Complete This Form” at the beginning of the instructions to provide a way to limit the physical and mental requirements of the respondent to track down those items or make it easier for the respondent to ask for help from others to complete the form. We agreed to make this revision, as revising this section on the form will assist in reducing the burden and stress on the respondents of remembering information after starting the form.

Change #4: We revised the language under the “YOUR MEDICAL RECORDS” section. We modified the language to provide clear instructions that involves medical records. We removed:

“If you have any of your medical records covering the last 12 months, send or bring them to our office with this completed report. Please tell us if you want to keep your records so we can return them to you. If you have a scheduled appointment for an interview, bring your medical records, your prescription medicine containers (if available), and the completed report with you. That you do not already have.”

Justification #4: The CCD mentioned that CDRs are costly to beneficiaries, who often need to pay for medical records or appointments with doctors and other providers to fill out the forms (and any transportation required to get to and from these appointments). Although some states require medical records to be provided free to Social Security disability claimants, this does not extend to beneficiaries undergoing CDRs. The individual does not need to obtain any medical records or contact their medical provider for assistance to complete the form. With the respondent’s permission, SSA will request their- medical records.

Change #5: We revised the information that was in the “FOR SSA USE ONLY” box. We removed all of the wording in this box except for the “Date of your last medical disability decision”.

Justification #5: Revising this information helps users identify a pertinent date that is needed to complete the questions throughout this form. During public comment period, some users stated that they had a difficult time remembering the date of their last medical review.

Change #6: We revised the title of SECTION 1. We removed: “THE DISABLED PERSON” from the title. The section now reads: INFORMATION ABOUT YOU.

Justification #6: We implemented changes to the title to remove “disabled person” from the form to provide clarity on the form when completing.

Change #7: We added language under the title of SECTION 1. We added the following language: If you are filling out this report for someone else, please provide information about him or her. When a question refers to "you" or "your", it refers to the person receiving disability benefits.

Justification #7: Multiple commenters, including the CCD, suggested the form needs to be written in Plain Language. They also stated that SSA should ensure the form is written in as clear and concise language as possible, and that SSA should analyze the form for literacy level. We are making this change to revise the language to Plain Language.

Change #8: We revised the language in question 1A and 1C. In question 1A we ask for a full middle name and any suffix. In question 1C, we removed: Have you used any other names on your medical or educational records in the last 12 months? We added the following language: In the last 12 months, have you used any other names on your medical or educational records? Also, in the examples we added “other names.”

Justification #8: Multiple commenters, including the CCD, suggested the form needs to be written in Plain Language. They also stated that SSA should ensure the form is written in as clear and concise language as possible, and that SSA should analyze the form for literacy level. We are making this change to revise the language to Plain Language.

Change #9: We added a new question as question 1E. We added the following language: Is your residence address the same as your mailing address?

□ YES □ NO – Complete below

Justification #9: Multiple commenters, including the CCD, suggested the form needs to be written in Plain Language. They also stated that SSA should ensure the form is written in as clear and concise language as possible, and that SSA should analyze the form for literacy level. We are making this change to revise the language to Plain Language.

Change #10: We revised the language in question 1.F. We added the following language: “so we can call and leave a message, if needed” and the request for a secondary number if available. We removed the checkbox: “Check this box if you have a phone or a number where we can leave a message.”

Justification #10: We received public comment and feedback from Usability testing that this question needed to provide more clarity as to which information is being gathered and for what reason. We revised this question to assist in cases that may involve whereabout unknown and additional methods for phone contact if necessary.

Change #11: We revised the heading of SECTION 2. We added the following language: “SOMEONE WE CAN.”

Justification #11: Based on public comment and Usability Testing revising the heading provides clarity of the content of this section.

Change #12: We added language under the heading of SECTION 2. We added the following language: “This section lets you give the name of a friend or relative (other than your doctors) we can contact who knows about your medical conditions, and can help you with your case. Examples include a family member, friend, or neighbor.”

Justification #12: Multiple commenters, including the CCD, suggested the form needs to be written in Plain Language. They also stated that SSA should ensure the form is written in as clear and concise language as possible, and that SSA should analyze the form for literacy level. We are making this change to revise the language to Plain Language.

Change #13: We revised the language in question 2B. We removed “Disabled” and added the following language: “Person in 1.A.”

Justification #13: Based on public comment and Usability Testing we revised this question to provide clarity of the information we are asking for from respondents.

Change #14: We removed questions 2F through 2J to SECTION 10 on the form.

Justification #14: Multiple commenters, including the CCD, suggested the form needs to be reformatted for fluency of the questions being asked throughout the form. Therefore, we are removing these questions.

Change #15: We consolidated Section 3 from the prior version of the paper form. We added all the questions pertaining to medical conditions, medical providers and medication into this section. We revised the heading of SECTION 3. We removed “Conditions” and added “Information” to the heading.

Justification #15: Multiple commenters, including the CCD, suggested the form needs to be reformatted for fluency of the questions being asked throughout the form. Based on public comment and Usability Testing revising the heading provides clarity of the content of this section.

Change #16: We added language under the heading of SECTION 3. We added the following language: “This section lets you share general medical information with us and assists with any records requests. We will use this information to see what additional questions or forms we may need to send you.”

Justification #16: Multiple commenters, including the CCD, suggested the form needs to be written in Plain Language. They also stated that SSA should ensure the form is written in as clear and concise language as possible, and that SSA should analyze the form for literacy level. Adding this information assists the users in understanding why the information is being asked in this section.

Change #17: We moved Question 3C to Question 3A. Question 3A now reads “List each physical and/or mental health condition that limits your ability to work separately. If under age 18, list the physical and/or mental health conditions that limit the child’s ability to do the same things as other children the same age.” If you need to list more conditions, use Section 9 – Remarks.

Justification #17: Multiple commenters, including the CCD, suggested the form needs to be written in Plain Language. They also stated that SSA should ensure the form is written in as clear and concise language as possible, and that SSA should analyze the form for literacy level. We are making this change to revise the language to Plain Language.

Change #18: We moved Question 3A to Question 3B. We removed the following language “(if not USA)”. Question 3B reads, “What is your height? _____ feet _____ inches OR __________ centimeters.”

Justification #18: Based on public comment and Usability Testing, we revised Question 3B to provide clarity on the information we are requesting from respondents.

Change #19: We moved Question 3B to Question 3C. We moved the following language “(if not USA).” Question 3C now reads, “What is your weight? __________ pounds OR __________ kilograms.”

Justification #19: Based on public comment and Usability Testing, we revised Question 3C to provide clarity on the information we are asking from respondents.

Change #20: We revised the language in Question 3D. We added the language, “Within the last 12 months” to the beginning of the question. We revised this question to only ask for the name of facility or doctor/phone number/address and last date seen. We removed the following language from the chart, “First Visit, Next Scheduled Appointment (if any), Emergency Room Visits and Overnight Hospital Stays.”

Justification #20: Based on public comment and Usability Testing, we revised Question 3D to provide clarity on the information we are asking for from respondents. Public comment suggested this was the most stressful question and caused the most burden when completing this form. We believe this revision will help alleviate some of this stress.

Change #21: We revised Question 3E by adding a table to collect the information within the question.

Justification #21: Based on public comment and Usability Testing this question was difficult to complete due to the free format narrative and creating unity within the form. Revising the question by adding a table provides a simply way to collect the information and reduces burden.

Change #22: We revised the language on Question 3F. We added the following language, “Within the last 12 months” to the beginning of the question.

Justification #22: Based on public comment in reference to the burden of gathering and remembering information, we revised this question to add clarity of the information we are requesting and to reduce any prior information the respondent might include.

Change #23: We revised Question 3G by adding a table to collect the information within the question.

Justification #23: Based on public comment and Usability Testing this question was difficult to complete due to the free format narrative and creating unity within the form. Revising the question by adding a table provides a simply way to collect the information and reduces burden.

Change #24: We added a new question as Question 3H. We added the following language:

Is the person listed in 1.A. under age 14?

□ NO (Go to Section 4)

□ YES (Go to Section 10).

Justification #24: Multiple commenters, including the CCD, suggested the form needs to be written in Plain Language. They also stated that SSA should ensure the form is written in as clear and concise language as possible, and that SSA should analyze the form for literacy level. We added this question to inform the individual completing the form that if the person with a disability is under age 14 that they could skip the rest of the questions in (Sections 4‑9) and go straight to Section 10 which is at the end of the form.

Change #25: We added new language under the heading of SECTION 4. We added the following language: “This section lets you tell us if you have worked. If we have any additional questions about your work we may contact you.”

Justification #25: Multiple commenters, including the CCD, suggested the form needs to be written in Plain Language. They also stated that SSA should ensure the form is written in as clear and concise language as possible, and that SSA should analyze the form for literacy level. Adding this information assists the users in understanding why SSA is requesting the information in this section.

Change #26: We revised the way that the information in Question 4A is collected. We added a table to collect the information about any work since the date of the last medical decision. We also added checkboxes to inquire which type of work to assist the field offices.

Justification #26: Multiple commenters, including the CCD, suggested the form needs to be written in Plain Language. They also stated that SSA should ensure the form is written in as clear and concise language as possible, and that SSA should analyze the form for literacy level. Adding the check boxes reduces the burden on the public from completing a free format narrative box.

Change #27: We added a new question as question 4B. We added the following language: Is the person listed in 1.A. under age 18?

□ NO (Go to Section 5)

□ YES (Go to Section 10)

Justification #27: Multiple commenters, including the CCD, suggested the form needs to be written in Plain Language. They also stated that SSA should ensure the form is written in as clear and concise language as possible, and that SSA should analyze the form for literacy level. We added this question to inform the individual completing the form that if the person with a disability is under age 18 that they could skip the rest of the questions in (Sections 5‑9) and go straight to Section 10 which is at the end of the form.

Change #28: On the prior version of the form, we changed SECTION 8 to SECTION 5. We added new language under the heading of SECTION 5. We added the following language: This section lets you share information about your participation in support services. Examples of support services can include:
- An Individualized Education Program (IEP) through a school (if a student age 18-21)
- An individualized work plan with an employment network under the Ticket to Work Program
- A Plan to Achieve Self-Support (PASS)
- An individualized plan for employment with a vocational rehabilitation agency or any other organization

Justification #28: Multiple commenters, including the CCD, suggested the form needs to be written in Plain Language. They also stated that SSA should ensure the form is written in as clear and concise language as possible, and that SSA should analyze the form for literacy level. Adding this information assists the users in understanding why SSA is requesting the information in this section.

Change #29: We revised the language in Question 5A and added a yes and no checkbox. The language in Question 5A now reads: Since the date of your last medical disability decision, have you participated or are you participating in any support services mentioned above or any other vocational rehabilitation, employment services, or other support services to help you to go work? See date on top of Page 3.

□ NO (Go to Section 6)

□ YES (Complete section below)

We removed the following language: If YES, what year did you last attend any school? When did you start participating in the plan or program?

Justification #29: We reworded the question to follow the format of the other questions within the form. In addition, adding a table to collect the information in this question will reduce the burden on respondents answering this question and provides clarity of the information needed.

Change #30: We revised the format of how the information is collected in Question 5B. We added a chart to collect the information. The question now reads: Are you still participating in the plan or program? (Select answer below)

□ YES -

Date began: ___________

MM/YYYY

Expected completion date: ___________

MM/YYYY

□ NO -

Date began: ___________

MM/YYYY

Reason stopped:

Date stopped: ___________

MM/YYYY

Justification #30: Based on public comment and Usability Testing this question was difficult to complete due to the free format narrative and creating unity within the form. We believe that adding a table to collect the information in this question will reduce the burden for respondents answering this question and provides clarity of the information needed.

Change #31: We revised the format of how the information is collected in Question 5C. We added a chart to collect the information. The question now reads: What types of services, tests, or evaluation were provided?

Select all that apply:
□ Vision test	□ Work classes		□ Other - Please explain:
□ Psychological/IQ test		□ Hearing test		□ Work Evaluation

Justification #31: Based on public comment and Usability Testing this question was difficult to complete due to the free format narrative and creating unity within the form. Revising the question by adding a table provides a simply way to collect the information reducing burden.

Change #32: We added language under the heading of SECTION 6. We added the following language: This section lets you share the contact information for anyone else or any other organization that may have medical information about your physical or mental health condition(s) that you did not list in Questions 3.D. or 5.A.

Justification #32: Multiple commenters, including the CCD, suggested the form needs to be written in Plain Language. They also stated that SSA should ensure the form is written in as clear and concise language as possible, and that SSA should analyze the form for literacy level. Adding this information assists the respondents in understanding why SSA requests the information in this section.

Change #33: We revised the language in Question 6. We removed: about your physical or mental condition(s) (including emotional and learning problems) covering the last 12 months. We added the following language: Examples include social services agencies, welfare agencies, attorneys, prisons, workers’ compensation, insurance companies who have paid you disability benefits. The question now reads: Within the last 12 months, does anyone else have your medical information about your physical or mental condition(s) (including emotional and learning problems) covering the last 12 months or are you scheduled to see anyone else? Examples include places such as social services agencies, welfare agencies, attorneys, prisons, workers’ compensation, insurance companies who have paid you disability benefits.

Justification #33: Multiple commenters, including the CCD, suggested the form needs to be written in Plain Language. They also stated that SSA should ensure the form is written in as clear and concise language as possible, and that SSA should analyze the form for literacy level. We are making this change to revise the language to Plain Language.

Change #34: We revised the title of SECTION 7 to EDUCATION, AND TRAINING, AND LITERACY. We also added language under the heading of SECTION 7. We added the following language: This section lets you share education, training, and literacy information with us.

Justification #34: Multiple commenters, including the CCD, suggested the form needs to be written in Plain Language. They also stated that SSA should ensure the form is written in as clear and concise language as possible, and that SSA should analyze the form for literacy level. Revising the title of the section provides clarity of the information that SSA collects in this section. In addition, adding a narrative of information assists the users in understanding why SSA requests the information in this section.

Change #35: We revised the wording in Question 7D. We added the word READING to the beginning of the question and bolded the word “read.”

Justification #35: In Question 7D we added the word reading before the question to ensure that question is being asked for separate reason than the Question 7E; and that the questions may appear the same, but they collect different information. Multiple commenters, including the CCD, suggested the form needs to be written in Plain Language. They also stated that SSA should ensure the form is written in as clear and concise language as possible, and that SSA should analyze the form for literacy level. We are making this change to revise the language to Plain Language.

Change #36: We revised the wording in question 7E. We added the word WRITING to the beginning of the question and bolded the word “write.”

Justification #36: In question 7E we added the word writing before the question to ensure the question was asked for separate reason than question 7D; and that the questions may appear the same, but they collect different information. Multiple commenters, including the CCD, suggested the form needs to be written in Plain Language. They also stated that SSA should ensure the form is written in as clear and concise language as possible, and that SSA should analyze the form for literacy level. We are making this change to revise the language to Plain Language.

Change #37: We added language under the heading of SECTION 8. We added the following language: This section allows you to tell us how your conditions affect your everyday life. This will help us further understand your medical conditions.

Justification #37: Multiple commenters, including the CCD, suggested the form needs to be written in Plain Language. They also stated that SSA should ensure the form is written in as clear and concise language as possible, and that SSA should analyze the form for literacy level. Revising the title of the section provides clarity of the information we collect in this section. Adding a narrative of information assists the users in understanding why SSA is collecting the information in this section.

Change #38: We have made questions 8A and & 8B optional for respondents to complete.

Justification #38: As per several public comments and our usability testing sessions, we noted that respondents have difficulty answering these open-ended Daily Living questions. While we are working to remove them from the form and screens, we will make them optional for respondents to answer in the interim.

Change #39: We revised the language in question 8C and revised the table to collect the information being asked in this question.

Justification #39: Multiple commenters, including the CCD, suggested the form needs to be written in Plain Language. They also stated that SSA should ensure the form is written in as clear and concise language as possible, and that SSA should analyze the form for literacy level. Revising the table in this question provides clarity of the information we collect in this section and allows space to provide any additional information.

Change #40: We revised the wording under SECTION 9. The narrative now reads: Please write any additional information you did not give in earlier parts of this report. If you did not have enough space in the sections of this report to write the requested information, please use this space to provide the additional information requested in those sections. Be sure to note the section (and question number) to which you are referring.

Justification #40: Multiple commenters, including the CCD, suggested the form needs to be written in Plain Language. They also stated that SSA should ensure the form is written in as clear and concise language as possible, and that SSA should analyze the form for literacy level. Revising the title of the section provides clarity of the information we collected in this section. Adding clarity to the narrative assists the users in understanding how to use this section to record any additional information that they would like to report.

Change #41: We removed question 10 from the prior form. We moved the question to SECTION 4, which is now question 4A.

Justification #41: Multiple commenters, including the CCD, suggested the form needs to be reformatted for fluency of the questions being asked throughout the form. Moving this question to SECTION 4 provides clarity of the information that we collect and helps with the form flow for information in this section.

Change #42: We added a new section to the form, which is SECTION 10. We removed the question from SECTION 1 asking “who is completing this form” to this new section. The format of this question is the same as it appeared on the prior version of this form.

Justification #42: Multiple commenters, including the CCD, suggested the form needs to be reformatted for fluency of the questions being asked throughout the form.

Terms of Clearance

OMB placed the following Terms of Clearance on this Information Collection when they last approved it on 6/28/21:

OMB advises the agency that based on the public comments received on docket SSA‑2018-0026 that reference this information collection, the burden estimate associated with this information collection is likely substantially too low.

Following approval of this ICR, the agency should work with OMB to develop and document a comprehensive qualitative and quantitative understanding of the burdens associated with this information collection. This documentation, which should be included as a component of ICR renewal in 12 months, must start with the process of selecting of the respondent, and record the typical experiences of a respondent through to the endpoint of adjudication (to include, if relevant, other information collections commonly involved during the Continuing Disability Review process). To the extent that burden varies by virtue of the distinct user segments who respond to this collection, the agency should strive to document a series of typical experiences. To achieve this, the agency (in coordination with OMB) should consult with both internal and external stakeholders and leverage modern customer research and stakeholder engagement techniques, to include journey mapping and user interviews.

Any digital instruments created for this information collection should undergo extensive user testing, at least some of which representatives from OMB should be invited to attend. The agency must also consult with OMB regarding the development of any digital instruments early-enough in product development so that OMB input on burden-minimization and other digital product best practices can be integrated into the final instrument. The documentation regarding that user testing should be included as an addendum to affirm that the digital product’s user experience is designed to be burden-minimizing.

SSA Responses:

OMB Term #1:

SSA Response: We implemented changes to the paper SSA-454 based on public comments, and revised our burden estimate for the SSA-454-BK based on the public comments we received on docket SSA-2021-0029 (our 60-day Comment Period Notice for this information collection request). Based on current management information data, and the comments we received, we are changing the burden estimate to 8 hours for respondents to read the instructions, gather the data, and complete the form. In addition, we also included the field office wait times (an average of 24 minutes) and the travel times to a field office (average of 30 minutes) in our Supporting Statement for those respondents who use the paper or EDCS modalities.

We are basing our burden estimate for the new i454 on the usability testing we conducted on the screens and on comments we received on docket SSA-2021-0029. Per our usability testing results, it took most respondents an average of 60 minutes to read the instructions, and complete and submit the screens. However, based on the public comments we received, we understand that it can take hours to collect the data necessary to complete the and submit the screens. Therefore, we are also using a combined burden estimate of 8 hours for the new i454.

OMB Term #2:

SSA Response:

SSA conducted usability testing on the new i454 screens, which gave us a better understanding of the burdens associated with this information collection. In addition, we invited OMB to attend these usability sessions held with external stakeholders to discuss the CDR process and the new i454 screens.

As part of this document, we are including our usability testing results. In addition, as per this Term of Clearance, we are still in the process of creating a comprehensive evaluation which explains the burdens for the following items:

The process of selecting the respondents
A record the typical experiences of various respondents through to the endpoint of adjudication (which includes a listing of other information collections commonly involved during the Continuing Disability Review process).
1. This record includes experiences from several types of respondents (i.e., individual adult claimants; child claimants; representative payees; etc.)
An accounting of our consultation with both internal and external CDR stakeholders
The results of the customer research and stakeholder engagement techniques, including journey mapping and user interviews

We will provide this document to OMB once we complete it. As this is a comprehensive document, we ask that OMB approve this information collection request prior to SSA’s submitting this document to OMB.

OMB Term #3:

SSA Response:

As reported above, we conducted usability testing on the new i454 screens to assess their use prior to implementation. During the 60-day comment period for the first Federal Register Notice, we also invited OMB to join us during our discussion sessions with the representative payee advocacy groups, as well as the ongoing usability testing, we completed for the i454 in August 2021. We welcomed OMB’s input on burden minimization and digital product best practices and integrated some of those into the final screens. As requested, we included the documentation of our usability testing (see the section below which includes our notes from those usability testing sessions).

Public Comments on the Information Collection

60-Day Comment Period Federal Register Notice (FRN):

The 60-day Comment Period began on August 18, 2021, and ended on October 18, 2021, at 11:59pm. We received a total of 101 public comments on the 60-day comment period FRN. Of those, we received and posted 99 comments on the CDR, and we did not post two comments (one duplicate comment, and a comment we received for a different information collection). The following contains a summary of those comments and SSA’s responses:

Overall Comments on the Time Estimate and Length of the SSA-454-BK

Comment #1: Multiple commenters questioned the time estimate, and stated it was underestimated by a large margin (in some cases of several hours to several days). They cited the need to read the instructions, gather the information, and complete the form each take longer than SSA’s one hour estimate (in some cases days or weeks). Many commenters suggest that SSA should adjust the burden estimate for this form to a number that more accurately shows the time it takes to read the instructions, gather the information, and complete the form (e.g., between 4 – 20 hours).

SSA Response #1: Based on the comments we received, SSA is updating the burden estimate for the SSA-454-BK to an average of 8 hours to read the instruction, gather the data, and complete the form. Since we understand that some commenters report a longer time estimate than 8 hours, SSA will take the necessary steps to evaluate and reassess the estimated burden to complete the SSA-454. We evaluated the form and worked with internal and external partners to revise the form by streamlining the questions where appropriate with the goal of reducing the overall burden of completing the form.

Comment #2: The Consortium for Citizens with Disabilities (CCD) agrees that the Social Security Administration (SSA) is required by Congress to perform periodic Continuing Disability Reviews (CDR) on recipients of Supplemental Security Income (SSI), or Title II Social Security benefits awarded on the basis of disability. However, they are concerned about the considerable burden SSA’s current CDR review process, including its use of the SSA-454-BK form, places on claimants and the public. They not only think that SSA grossly underestimates the burden that responding to SSA-454-BK places on claimants and the public when it suggests the average burden is 60 minutes, but the CCD also believes SSA does not adequately consider the burden each inquiry places on claimants. Rather the CCD thinks SSA seeks some information for which the burden on the claimant far outweighs the utility of the answers to the questions. The CCD believes that if SSA considers the complete time burden spent by the claimant and third parties to review SSA‑454-BK, collect documentation, complete, and transmit this form, it would take an average of 15-20 hours. This would include:

times spent by the claimant receiving and reviewing the letter;
time spent by assisters (neighbors, family, community assisters and sometimes SSA claims representatives) helping the claimant understand the SSA-454-BK form, and the steps required to respond (particularly in cases where SSA knows the claimant has intellectual, cognitive, behavioral or language deficits);
time spent to collect information or documentation needed to complete the form;
time spent by medical, behavioral health, and other providers furnishing documentation and or fielding specific questions necessary to complete the form;
time spent securing assistance from advocates or lawyers;
time spent to actually complete form;
and the time required to transmit the SSA-454-BK to SSA.

The CCD notes that it is hard to give a precise estimate of what the real burden of completing an SSA-454 is because so many people completing it are doing so unassisted. They assert the time estimate needs to account for the items listed above, as well as time spent calling SSA to get assistance from claims representatives understanding these forms, which often includes getting through long wait times on the phone. They hope SSA will revise the burden estimate and take steps to reduce the Time Tax this process imposes on claimants. They reiterate that they believe the burden created by the CDR process should receive heightened scrutiny because this is a process that is being imposed exclusively on a population that has already been identified to have severe impairments.

SSA Response #2: Based on the comments we received, SSA is updating the burden estimate for the SSA-454-BK to an average of 8 hours to read the instruction, gather the data, and complete the form. However, we note that the CCD suggests a revised time estimate of 15-20 hours based on the seven items the CCD listed in their comment. To that end, SSA notes that the individual does not need to obtain any medical records or contact their medical provider for assistance to complete the form. With the respondent’s permission, SSA will request their- medical records. If the individual cannot complete the form, a Social Security Representative will assist them. Even so, we understand that the medical information is only a small part of the overall issue which the CCD notes for the burden estimate (i.e., reviewing the letter; time spent by those assisting the respondent; time spent securing an advocate/lawyer). To that end, we conducted an evaluation of the form and worked with internal and external partners to revise the form by streamlining the questions and instructions where appropriate with the overall goal of reducing the burden of completing the form.

Comment #3: The CCD also commented on the length of the form, stating that just completing the SSA-454-BK form is burdensome in and of itself. It is 15 pages long and requires multiple stamps to be mailed back to SSA. It requires beneficiaries to write short essays in response to questions, report all the medication they take and all of the medical treatment and providers they attend, and all of their daily activities. For adults and children with disabilities, this is usually a huge amount of information. The SSA-454-BK form asks for detailed summaries of the medical treatment received over the last 12 months, including the dates of first and last appointments, information that the individual themselves is unlikely to know in the detailed required to respond, and necessitating assistance from health care professionals and other service providers. While it would be challenging and time-consuming for anyone to fill out, many of those who will need to fill it out have disabilities that will add additional complexity. In addition, most claimants are unassisted and have huge challenges completing these forms due to symptoms of their impairments including poor memory, poor concentration, or other limitations. It is often most challenging for clients to remember specific details about their medical treatment called for by the form (date of first or last appointment and each test performed) which is concerning because it is the most important part of the forms showing continued disability.

SSA Response #3: SSA agrees with the CCD that this form used to collect the necessary information for a CDR is lengthy. To address that issue, we conducted an evaluation of the form and worked with internal and external partners to revise the form by streamlining questions and instructions where appropriate with the overall goal of reducing the burden of completing the form (see our revisions in the first section of this document, above).

In addition, we would note that we include a self-addressed, stamped envelope with Form SSA-454, our CDR form, so the respondent should not incur a cost in returning the completed form to us. We are also releasing a new web-based modality of the form called the i454 to increase accessibility and will allow the individual to submit the form online. In addition, respondents may complete the CDR forms via personal interview with a claims representative via telephone, which removes the need for them to complete the paper form and return it to SSA. Finally, we do not require individuals to complete all the sections, only the sections for which they have relevant information. Since we understand that may not be clear to the respondents, as part of our evaluation of the form, we have worked to address this issue.

Comment #4: Multiple commenters mentioned that the lengthy CDR form and process, as well as the fear of losing their SSI or SSDI payments if they fill out the form incorrectly, induces stress, making it more difficult for them to complete the form.

SSA Response #4: We follow our statutory obligation to periodically review all individuals for continuing eligibility for disability benefits, including individuals with permanent impairments. We acknowledge stress may occur due to medical reviews, but we are required to conduct them. We also understand that some may fear losing their SSI or SSDI payments if they fill out the form incorrectly. Our policy requires that we make sufficient attempts to contact individuals to collect the necessary information to complete the review. We also provide accommodations to individuals who require additional help (for example, individuals who are homeless, children, and individuals with mental disorders). We evaluated the form and worked with internal and external partners to revise the form by streamlining the questions where appropriate with the goal of reducing the overall burden of completing the form.

Comment #5: Several commenters mentioned that the form is too long and too involved for those respondents who are permanently or severely disabled, or for those with intellectual or developmental disabilities. These commenters also mentioned that the form can be physically painful to complete for those with certain physical disabilities, as it is very long and requires many hours of sitting and writing or typing. They believe SSA should not require a CDR from those who have medically proven disabilities that will not improve (e.g., lifelong disabilities, or degenerative diseases). They also believe that SSA should simplify the form and shorten it for those who need to complete it.

SSA Response #5: Although all respondents to the SSA-454 have previously been found to have a disability, we must follow our statutory obligation to periodically review all individuals for continuing eligibility for disability benefits, including individuals with severe or permanent impairments. To address the concern that completing the form can be physically painful to complete, we evaluated the form and worked with internal and external partners to revise the form by streamlining the questions and reducing narrative responses where appropriate with the goal of reducing the overall burden of completing the form.

Comment #6: One commenter stated that the fields for the short answer sections of the form are too short and too small to include all of the information requested. In addition, the remarks section, which the form states respondents can use to include more data is unlined and too short as well. This commenter suggests that SSA either expand these sections or eliminate them.

SSA Response #6: We conducted an evaluation of the form and worked with internal and external partners to revise the form by streamlining the questions and instructions where appropriate with the overall goal of reducing the burden of completing the form. We considered revisions and simplifications to these sections to adjust the format of these sections as appropriate.

Comment #7: Several commenters noted that the form includes multiple narrative questions regarding the claimant’s medical treatment, including many details which the respondent or those assisting them may not know without assistance from treating health care providers, requiring the respondents to request that information from the providers. As SSA also requests the medical evidence from the health care providers, it should not be necessary for the respondent to complete these sections of the CDR.

SSA Response #7: To determine if someone continues to have a disability, we must collect the names of their medical sources to request the medical records. We understand that much of the information we currently request on the SSA-454 may also be in the medical records. With this in mind, we evaluated the form and worked with internal and external partners to revise the form by streamlining the questions where appropriate with the goal of reducing the overall burden of completing the form. We considered revisions and simplified these sections to reduce the amount of specific medical information needed, dates of appointments or tests, and narrative responses as appropriate.

Comment #8: The CCD also mentioned that CDRs are costly to beneficiaries, who often need to pay for medical records or appointments with doctors and other providers to fill out the forms (and any transportation requited to get to and from these appointments). Although some states require medical records to be provided free to Social Security disability claimants, this does not extend to beneficiaries undergoing CDRs. Beneficiaries may need to hire representatives to assist them in completing the CDR paperwork.

SSA Response #8: Under our rules we will pay physicians not employed by the Federal government and other non-Federal providers of medical services for the reasonable cost of providing us with medical evidence. If beneficiaries’ medical sources will not provide sufficient medical evidence to make a medical determination, we may ask a beneficiary to attend an examination, also at our expense. Instructions on page one of the forms, also state that the individual does not need to obtain any medical records. With their permission, we will request their records. Additionally, the instructions state the individual should not ask their healthcare provider to complete the form. If an individual cannot complete the form, a Social Security Representative will assist them free of charge. We provided additional clarifying instructions that the beneficiary is not required to provide or request medical records from their health care providers.

Overall Comments on the Complexity of the CDR Process and Stress on the Respondents

Comment #9: The CCD commented that everyone undergoing a CDR, by definition, has already been found to have a severe condition that is disabling for at least a year, if not terminal. In many cases, the impairments that prevent beneficiaries from working also prevent them from accurately completing the SSA-454-BK (either on paper or online). In addition, disability beneficiaries are often older and have lower income, less stable housing, and less education than the general population, providing additional challenges when they need to fill out the SSA-454-BK and submit supporting documents like medical records. For children undergoing CDRs, the burden on families and service providers is substantial – adults must take time off of work and children must take time out of school for medical appointments in response to the form. These all contribute to the stress and burden on the respondents.

SSA Response #9: We follow our statutory obligation to periodically review all individuals for continuing eligibility for disability benefits, including individuals with permanent impairments. We acknowledge stress may occur due to medical reviews, but we are required to conduct them. We evaluated the form and worked with internal and external partners to revise the form by streamlining the questions where appropriate with the goal of reducing the overall burden of completing the form. We considered revisions and simplified these sections to reduce the overall burden. We believe that simplifying the form, reducing the amount of information collected, and using plain language, will help beneficiaries to accurately complete the form, and address the challenges to complete the form due to income brackets and lower education levels.

Comment #10: One commenter mentioned that it is more difficult to fill out the CDR form via telephone, and it would be easier to fill out the CDR form in a field office. However, the commenter noted it is not possible to do that right now due to COVID, which adds to the stress of filling out the form.

SSA Response #10: Telephone appointments are available to assist respondents to complete the form. Since these comments were made, SSA has resumed in person office visits to assist the public. We evaluated the form and worked with internal and external partners to revise the form by streamlining the questions where appropriate with the goal of reducing the overall burden of completing the form. We are also releasing a new Internet modality called the i454 to increase accessibility.

Comment #11: Several commenters suggested that the SSA developed the CDR form to be overly complex so that respondents will fail at completing it, requiring them to go through the appeals process. They suggest that SSA either simplify the process, possibly by asking only whether someone still has a disability, or remove the CDR process entirely.

SSA Response #11: We follow our statutory obligation to periodically review all individuals for continuing eligibility for disability benefits, including individuals with permanent impairments. We understand that some individuals may have difficulty complying with our procedural requirements. As such, we enhanced our policies to further assist individuals with meeting program requirements. For example, we will contact beneficiaries at their address of record when we initiate a CDR and allow multiple opportunities to provide information. Our disability determination services (DDS) use similar procedures when an individual does not cooperate with requests for information, allowing multiple opportunities to provide information that is necessary to make a medical determination. Our policy requires that we make sufficient attempts to contact individuals. We also provide accommodations to individuals who require additional help (for example, individuals who are homeless, children, and individuals with mental disorders). Our protective process, by design, affords the beneficiary the multiple opportunities to complete the CDR.

Comment #12: One commenter suggested that SSA needs to develop a CDR system that is easier to use and asks fewer questions so that claimants can fill it out on their own without needing to ask for help.

SSA Response #12: We conducted an evaluation of the form and worked with internal and external partners to revise the form by streamlining the questions and instructions where appropriate with the overall goal of reducing the burden of completing the form.

Comment #13: One commenter suggested that SSA should send multiple notifications in multiple ways (my USPS mail, via email, via telephone, through the mySocial Security accounts, etc.) to the CDR recipients, both before sending the CDR and after sending it, to ensure recipients know they received a request to complete the CDR.

SSA Response #13: We contact beneficiaries or recipients at their address of record when we initiate a CDR and allow multiple opportunities to provide information. Our policy requires that we make sufficient attempts to contact individuals. We also provide accommodations to individuals who require additional help (for example, individuals who are homeless, children, and individuals with mental disorders). Our protective process, by design, affords the beneficiary the multiple opportunities to complete the CDR.

Comment #14: One commenter stated that SSA ignores requests for accommodations to help complete the CDR; however, SSA has listened to requests for accommodations when it comes to how the respondents receive notices. This commenter wants to know why SSA is unable to accommodate similar request for completing the CDR form, or at least creating a way for respondents to complete the CDR on their own with no help.

SSA Response #14: If an individual cannot complete the form, a Social Security Representative will assist them. Individuals can receive assistance by telephone or in office appointments. We are also releasing a new Internet modality called the i454 to increase accessibility. We conducted several usability testing sessions with members of the public, and members of the advocacy community for the i454. We used this feedback to address concerns regarding the burden of the form.

Comment #15: Several commenters stated that people living with mental health conditions often experience significant stress in reaction to the receipt of official government documents and may lack the capacity to respond immediately. They noted that since the CDR form is so long and dense, that often those with mental health conditions have difficulty responding to it, or choose not to respond at all, causing a cessation of much needed benefits. They suggest SSA take steps to simplify this form and reduce the burden on beneficiaries and those who help them complete it.

SSA Response #15: Our policy requires that we make sufficient attempts to contact individuals. We also provide accommodations to individuals who require additional help (for example, individuals who are homeless, children, and individuals with mental disorders). Additionally, we reviewed the current form to identify areas to simplify and streamline the form and reduce the overall burden to complete the form.

Comments on the Frequency of CDRs

Comment #16: Several commenters asked that SSA not increase the frequency of CDRs, stating that the process is onerous and burdensome, especially on vulnerable populations with lifelong disabilities and degenerative disorders, and an increase in the frequency would be unfair and pointless.

SSA Response #16: SSA is not proposing to increase the frequency of CDRs.

Comment #17: The CCD recommends that SSA revisit and improve processes related to ensuring that claimants diaried for CDR reviews receive the proper paperwork regarding their review. They note that many claimants report that they do not receive CDR paperwork in a timely manner, or at all. Per the CCD they believe that SSA sometimes sends these forms to an incorrect address because the claimant has moved or SSA has not properly updated the address in its numerous databases. They assert that this causes considerable problems, because if someone does not respond to the SSA-454-BK, it can lead to benefit cessation. Per the CCD, some of their members report that SSA does not always receive and process the SSA‑454-BK when the claimant sends them in, and some claimants will send in forms repeatedly, while others will have their benefits terminated because they do not realize SSA did not receive their SSA-454-BK. The CCD’s members have also identified receipt of the form as a common pain point for claimants. They say the fact that beneficiaries do not receive the CDR forms is also supported by SSA data showing 40,000 beneficiaries are terminated from SSI each year because their “whereabouts [are] unknown: (e.g., returned mail) or because they “failed to furnish [a CDR] report.” The CCD recommends that SSA revisit and improve processes related to ensuring that claimants identified for CDR reviews receive their proper paperwork.

SSA Response #17: When a CDR is initiated, we forward the notice, and associated forms to the current address we have in our records. To ensure that the beneficiary or recipient received the forms, we follow up through various methods (telephone, mail, third-party contact) to reach the beneficiary. If the beneficiary reports they did not receive the CDR paperwork, we verify the address, and re-send for completion. If we are unable to reach the beneficiary, we follow a series of policies to further assist individuals with meeting program requirements. For example, we contact beneficiaries at their address of record when we initiate a CDR and allow multiple opportunities to provide information. Our disability determination services (DDS) use similar procedures when an individual does not cooperate with requests for information, allowing multiple opportunities to provide information that is necessary to make a medical determination. Our policy requires that we make sufficient attempts to contact individuals. We also provide accommodations to individuals who require additional help (for example, individuals who are homeless, children, and individuals with mental disorders). Our protective process, by design, affords the beneficiary multiple opportunities to complete the CDR. Once we fully implement the new web-based i454, we will also be able to send a message to an individual with a my SSA account, notifying them they can complete the necessary CDR forms online, including the new i454. This new modality will increase the accessibility for beneficiaries and recipients, and will allow respondents to submit the forms online, and not through the mail.

Comments on Improving Form SSA-454-BK

Comment #18: Multiple commenters, including the CCD, suggested the form needs to be written in Plain Language, as it is currently difficult to read, and that it would also be helpful for the form to include pictures for those who are unable to read. They also stated that SSA should ensure the form is written in as clear and concise language as possible, and that SSA should analyze the form for literacy level. These commenters also suggest that SSA conduct outreach via telephone for claimants that SSA is aware are illiterate.

SSA Response #18: We agree that the form should be in Plain Language. We conducted an evaluation of the form to revise the form as appropriate by streamlining the questions and instructions with the overall goal of reducing the burden of completing the form. During this evaluation we simplified the questions where appropriate and utilized plain language to increase readability.

Comment #19: One commenter suggested SSA add a list of “What You Need to Complete This Form” at the beginning of the instructions to provide a way to limit the physical and mental requirements of the respondent to track down those items or make it easier for the respondent to ask for help from others to complete the form.

SSA Response #19: We considered additional instructions to assist in the preparation to complete the form. We conducted an evaluation of the form and worked with internal and external partners to revise the form by streamlining the questions and instructions where appropriate with the overall goal of reducing the burden of completing the form.

Comment #20: One commenter suggested SSA should develop a new version of the paper form with input from those who work in usability, and include feedback and input, as well as usability testing, from a number of people with a range of abilities and disabilities who can provide comments on how to better improve the process.

SSA Response #20: We conducted several usability testing sessions with members of the public, and members of the advocacy community for the i454 and paper SSA-454. We used this feedback to address concerns regarding the burden of the form. We also conducted an evaluation of the form and worked with internal and external partners to revise the form as appropriate by streamlining the questions and instructions with the overall goal of reducing the burden of completing the form.

Comment #21: Multiple commenters suggest that SSA translate the SSA-454-BK into other languages for people who do not speak English.
- One commenter notes that the Language Access Plan from fiscal year 2018-2019 identifies the following goals: delivering services effectively, improving the way SSA does business, and ensuring stewardship. The commenter reiterated that since this form is only available in English, a person who does not speak English or only speaks limited English needs to get help from another person read and respond to the questions. The commenter notes this is an added layer of difficulty and translation from English to another language alone can double the amount of time spent completing the document. The commenter states the Language Access Plan says that the agency regularly reassesses which documents should be translated. The commenter suggests that SSA assess this form for translation into multiple languages now.
- The CCD asserts that SSA should make the form as accessible as possible, as it is being sent exclusively to claimants who have already been adjudicated to have a severe disability. Therefore, the CCD recommends SSA make this form available in multiple languages. The CCD notes that SSA collects information about literacy and limited-English proficiency status as part of its disability adjudications. They emphasize that through that information, SSA must be aware that many of its claimants who have been found eligible for SSA benefits are limited English proficient, or lack literacy, and are not able to read these critical forms. Because the ability to respond to this form could lead to a cessation of benefits, the CCD believes it essential that SSA provide these in a language the claimant can read, whenever possible.

SSA Response #21: SSA will provide an interpreter free of charge to individuals requesting language assistance or when it is evident that such assistance is necessary to ensure that the individual is not disadvantaged. Question 1.G. of the current SSA-454 also states “If you cannot speak and understand English, we will provide an interpreter free of charge.” Respondents can find additional information regarding people that need interpreters at https://www.ssa.gov/multilanguage/langlist1.htm. While this form does have a Spanish language version used in Puerto Rico, we are not currently able to make this form available in multiple languages, which is why we provide free interpreter services. We are also exploring adding an insert which translate the “Need help?” information in five common languages. The insert will also include the link above to SSA’s multilanguage list.

Comment #22: The CCD recommends that SSA take steps to reduce the burden on claimants by truncating and streamlining the SSA-454-BK. They suggest that SSA afford specific consideration to the utility of each piece of information solicited as well as the burden each question places on the claimant. The CCD notes that although detailed medical information is no doubt useful to evaluating ongoing disability claims, some of this information is not absolutely necessary to adjudicating the claim but may place a large burden on the responder. They recommend removing any questions that are not absolutely necessary to initiate a CDR review or including modifiers to make clear that this information is not required if it is not known.

SSA Response #22: We conducted an evaluation of the form to revise the form as appropriate by streamlining the questions and instructions with the overall goal of reducing the burden of completing the form. For example, we are proposing to reduce the amount of information collected in Section 4 – Medical Treatment, by simplifying the collection of medical source information to the name of the health care provider, telephone number, address, and date last seen (if known). Since we are specifically requesting sources who have medical records for the past 12 months, we also propose to remove the need for the beneficiary to provide dates of appointments or tests. In addition, we will only collect this information in one place on the form instead of repeating it on multiple pages. We are also simplifying, the information collected in Section 9 – Daily Activities. We made these revisions in response to the CCD’s comments.

Comments Regarding Specific Revisions to Form SSA-454-BK

Comment #23: The commenter suggests that SSA revise Pages 1-2 of the instructions to include a statement with contact information for how to request assistance in completing this form by contacting an agency interpreter for the benefit of a person with a disability or individual helping a person with a disability who cannot speak, read, write, or otherwise understand English. This commenter claims to understand that SSA policy states that any notice sent to a Limited English Proficiency (LEP) individual requiring contact regarding a claim must include the following statement: “We provide free interpreter services to conduct your Social Security business. However, if you prefer to have your own interpreter, you may do so, but with the understanding that our own interpreter may be present.” Even so, the commenter notes that many of their clients have barriers to accessing benefits, in part, because they lack an understanding of English. While the commenter understands that SSA does have policies to govern the use of interpreters, they believe a lack of an interpreter to translate notices and other documents (such as the CDR form) will continue to be an unnecessary hurdle for individuals with Limited English Proficiency (LEP).

SSA Response #23: We conducted an evaluation of the form and worked with our CDR product discovery workgroup, and with other internal and external partners to revise the form by streamlining the questions where appropriate with the goal of reducing the overall burden of completing the form. We added the SSA 1-800 number contact information to the form instruction page under the section titled “IF YOU NEED HELP.” We also added the statement “If you cannot speak and understand English, we will provide an interpreter free of charge” to this section as well. We also retained the current instructions on the form under Section 1: #1.G. that states, “If you cannot speak and understand English, we will provide an interpreter free of charge.”

Comment #24: One commenter suggested the following revisions to Section 1, Information About the Disabled Person, Pages 3:
- Question 1E: The commenter suggests removing the box asking if the individual with a disability has a phone number where SSA can leave a message, as the commenter feels this is unnecessary information.

SSA Response #24: We evaluated the form during an internal discovery session and reworded the question to remove the check box to indicate if SSA can leave a message. The revised language states: “DAYTIME PHONE NUMBER (Include Area Code) so we can call and leave a message, if needed. “(Include IDD and country code if you live outside the USA or Canada.)

Comment #25: Two commenters suggested the following revisions to Section 2, Contacts:

One commenter stated that in Section 2 – Contacts it is important to make it clear that listing a person who knows about your condition is optional. Many respondents simply do not have anyone in their life who would feel comfortable answering questions with the level of detail included in the Third-Party Function report.

A second commenter suggested moving Pages 3-4 of Section 2 to the end of the form, prior to Section 11, as the commenter notes that since this information is seeking the author of the form, as well as contacts if SSA needs additional information from other sources, it makes more sense to move it closer to the end of the form.

SSA Response #25: We reviewed all sections to ensure they are placed appropriately. We conducted an evaluation of the form and completed an internal CDR product discovery workgroup to revise the form by streamlining the questions where appropriate with the goal of reducing the overall burden of completing the form. We use the “Contact” information to assist with being able to reach the beneficiary if they are not available. This information could also be helpful in moving the CDR process along in the event that the individual is not available for recontact if necessary.

We agree that the section in “Contacts” seeking the author of the form should be moved to the end of the form for better placement.

Comment #26: One commenter suggested the following revisions to Section 3, Medical Conditions, Pages 4: The commenter suggests inserting the following in the paragraph for Question (3A): “list the physical and/or mental health conditions….” This commenter feels the addition of the word “health” will clarify the language better for respondents.

SSA Response #26: We conducted an internal evaluation of the form and completed a CDR product discovery workgroup, to revise the form by streamlining the questions where appropriate with the goal of reducing the overall burden of completing the form. We inserted the word “health” for additional clarity.

Comment #27: Multiple commenters suggested revisions to Section 4, Medical Treatment:
- One commenter recommended replacing this section with something shorter and more concise, similar to the HA-4631, which allows the recipient to provide the information DDS needs to gather the medical records. The commenter noted that those medical records will include the information requested in the current version of Section 4 and significantly reduce the time it takes to complete the form, as the respondent would not need to complete the section themselves (with the exception of submitting the information regarding the medical provider who can supply the medical records).

SSA Response: We removed the need for beneficiaries to provide detailed information regarding their providers, including dates seen and future appointments. The form captures the name of the healthcare provider/facility, the phone number, address, the condition treated, and the date last seen (MM/YYY) if known. This will greatly reduce the amount of information the beneficiary needs to provide.

Another commenter recommended the following revisions to Pages 5-11, Section 4: The commenter recommends distilling this section substantially by asking the individual to just provide the names of any health care providers (with a list of examples including a doctor, hospital, clinic, or other health care professional) they have visited in the last 12 months along with that health provider’s contact information (i.e., phone and address). This commenter notes that this is the most cumbersome and time-extensive portion of the form, and the section respondents have the most difficulty answering. They state that SSA can request a copy of medical records with the individual’s permission and should request a copy from each entity listed in the CDR. By reducing this Section, an individual with a disability does not have to try and remember all of the dates or tests conducted during the 12-month period. The entities identified should have all of this information on file.

SSA Response: We reformatted this section and provided additional examples of health care providers to include a doctor, hospital, clinic, or other health care professional.

The CCD recommended the following changes to Section 4: The CCD believes that Section 4, where the form asks for details about Medical Treatment, should be streamlined and truncated. The CCD speculates that SSA is requesting information about recent medical treatment to solicit medical records but believes Section 4 requests far more information than is necessary to complete this task. The CCD recommends significantly streamlining this section as follows:
- Questions 4(C), 4(D), and 4(E): Instead of asking for medical information in response to the same question three separate times (“Tell us who may have medical records covering the last 12 months about any of your physical or mental conditions(s) . . .”), the CCD recommends including only one question seeking treatment information stating: “Have you gone to see any doctors, psychiatrists, nurse practitioners, therapists, physical therapists, or other medical professionals in the last 12 months?” followed by a check box for yes or no. Then, in a second question, ask, if yes, where? Under the “if yes, where,” the CCD recommends SSA provide a few lines asking for the name of the institution, address and telephone number. This will allow SSA to get the necessary information it needs regarding recent treatment in a much more streamlined fashion. It will also significantly reduce the overall length of the form.

SSA Response: We revised the question asking if they have seen any health care professionals with a yes or no block. If yes, the beneficiary will complete the next section by providing the name(s) of the healthcare provider/facility, the phone number, address, the condition treated, and the date last seen (MM/YYYY) if known. We provide five entry spaces.

On Pages 5, 6, 8, and 10, the CCD also recommends SSA remove the request for the claimant to identify the first, last, and next appointment with each provider throughout the form. The CCD has found that most claimants do not know their scheduled appointments with this level of detail. The CCD notes that many claimants get upset when they do not know this information, and some get deterred and stop completing the form (leading to cessation of eligible individuals) when facing these obstacles. Others spend a significant amount of time contacting medical offices or pouring through their medical records to try to find these dates, creating additional burden. Although knowing dates of treatment may be helpful to SSA in making targeted medical requests, the CCD feels it is not necessary. The CCD believes that most medical record requests only require the name of the institution. Thus, any benefit garnered by getting responses to these specific questions is outweighed by the burden it puts on the claimant.

SSA Response: We removed multiple questions asking the beneficiary for the dates of their appointments (past and future) and reduced it to one question of date last seen (if known), in MM/YYYY format. We removed questions asking for the dates of testing.

The CCD requests SSA either remove or add the modifier “(if known)” after all questions soliciting the name of the healthcare professional that provided treatment in Section 4. The CCD states they found that many claimants do not know the name of the medical provider who treated them, sometimes because providers serve them as a team, or the claimant cannot recall their name(s). The CCD reiterates that many claimants experience stress when they realize they cannot provide this information and, in some cases, may stop completing the form. Per the CCD, in other cases, claimants may spend a significant amount of time collecting this information. The CCD finds that this level of detail is not required for medical records requests; thus, the CCD believes the burden caused to claimants outweighs its utility to SSA.

SSA Response: We will still need to collect the names of the healthcare provider to fully develop the necessary information to conduct a full medical review, with a complete record. We expanded the definition of “health care provider” to include a doctor, hospital, clinic, or other health care professional. If we do not collect this information on the SSA-454, we will not be able to obtain the medical records for the CDR and may request that the beneficiary would need to attend a consultative examination (CE) to obtain medical records.

The CCD recommends revising the question about hospital and emergency room visits, Question 4(D), to say the following: “Have you been treated at a hospital, an emergency room, or urgent care in the past twelve months?” followed by a yes/no check box. Following this question, the CCD suggests adding “if yes, where?” followed by lines asking for the name and address of the institution. At the end of each line the CCD asks SSA to include a check box for – “overnight stay.”

SSA Response: We removed the specific section regarding hospital and emergency room visits. We also reformatted the section to collect any healthcare provider (doctor, hospital, clinic, or other health care professional), and only collect the date last seen (if known).

The CCD also recommends SSA revise and consolidate the questions about medical testing throughout Section 4. The CCD notes that the paper SSA-454-BK asks claimants to report testing in multiple places. They recommend keeping the check boxes where claimants can indicate what sort of testing they have had, or have, scheduled, but think asking this question multiple times can be overwhelming for claimants. Instead, the CCD recommends including it only one time on the form. They also recommend SSA remove the sentence “Please give the dates for past and future tests” from the instructions (See, p. 6), or at a minimum, include the modifier “(if known).” They would further recommend adding the same modifier, “(if known)” after the phrase “Date of Test(s)” so it is clear that claimants do not need to provide that information if that question is not removed.

SSA Response: We revised the questions regarding testing to only appear on the form one time. We will ask the beneficiary to check the box next to the test(s) that were ordered within the last 12 months from the providers that they previously listed. We will not ask for any dates of the tests performed, or the future date of a test.

The CCD recommends removing the solicitation for the dates of these tests, because many claimants struggle with that information, and instead they suggest using that space to ask, “where did the test occur (if known).” Per the CCD, removing questions about the dates of testing, would be an important improvement to the form for the respondents. As they noted previously, many claimants are not aware of this information, and when confronted by these questions, claimants experience anxiety, sometimes stop completing the form, or spend considerable time trying to learn it. While the CCD agrees that treatment dates have some utility, they do not believe having these testing dates is particularly important to SSA’s reviewing efforts and their utility is outweighed by the burden they put on claimants.

SSA Response: We revised this section to eliminate the request for dates of the tests performed, or the future date of a test.

Comment #28: The CCD suggested revisions to Section 5, Medicines: The CCD understands SSA’s need to know what medications claimants are taking. However, the CCD notes that many claimants are unaware of which doctors prescribe certain medications, or the use of the medications. Further, the CCD states that some claimants spend considerable time trying to collect this information, but this information is easily gleaned from medical records and is not necessary for records request. Therefore, the CCD recommends adding the modifier “(if known)” after “If Prescribed Give Name of Doctor” and “Reason for Medicine.”

SSA Response #28: We conducted a CDR product discovery workgroup to revise the form by streamlining the questions where appropriate with the goal of reducing the overall burden of completing the form. We collect this information to assist the disability examiners to identify any other conditions, or possible medical sources that may have been inadvertently omitted from the medical source collection question on the form. We do provide the suggestion that the beneficiary can use their prescriptions or prescription medicine containers to obtain this information, since this information is on the prescription medication label. Also, on page 1 of the form in HOW TO COMPLETE THIS REPORT it states, “If you cannot remember the names and addresses of your health care providers, you may be able to get that information from a telephone book, Internet, medical bills, prescriptions or prescription medicine containers. If you do not know an answer, or the answer is “none” or “does not apply,” please write: “don’t know” or “none” or “does not apply.”

Comment #29: One commenter suggested revisions to Section 6, Other Medical Information: This commenter suggested that SSA move Section 6 to the end prior to Section 11. This commenter notes that since this information is seeking contacts if SSA needs additional information from other sources, it makes more sense to place it at the end of the form.

SSA Response #29: We revised the introduction to this section to explain the type of information we need to collect. The introduction now states: Please provide the contact information for anyone else or any other organization that may have medical information about your physical or mental health condition(s) that you did not list in Questions 3.D. or 5.A. We also provided examples: Examples include places like social services agencies, welfare agencies, attorneys, prisons, workers’ compensation, insurance companies. This section is collecting additional sources other than health care providers and now comes after the collection of Support Services.

Comment #30: One commenter suggested revisions to Section 7, Education and Training; Section 8, Vocational Rehabilitation, Employment, or Other Support Services; and Section 10, Work: This commenter notes that all three section on Pages 12-13, and 15, Sections 7, 8 and 10 are seeking information related to education, training, employment, and other support services, Therefore, the commenter recommends consolidating these questions into one section, and removing some of the questions that they deem duplicative of questions in other sections. For instance, this commenter suggests removing Questions 7(C) – 7(E) on Page 13, as they believe the form already addresses these in Section 1.

SSA Response #30: These sections collect different information regarding each section category. Questions 7.C. – 7.E. are based on regulation, and we use them to evaluate literacy as it relates to Education and Training (20 CFR 404.1564 and 20 CFR 416.964. We use the questions in Section 1 (1.G. – 1.I) regarding language to identify the language spoken as it relates to the need of an interpreter, based on SSAs LEP policy. Due to the nature of the information we gather in these sections, we could not combine them.

Comment #31: Multiple commenters suggested revisions to Section 9, Daily Activities:
- One commenter suggests eliminating Section 9 entirely, as they believe a continuing disability review should compare a person’s condition at the time of the review to their condition at the last time the condition was reviewed. The commenter asserts that for many respondents, this means comparing their condition to what it was at the time of application, where this information is not collected. Moreover, the commenter notes that a finding of disability does not require a person to be an invalid and completely unable to care for themselves. Per the commenter, a person who is able to do the tasks listed could still have a condition that prevents them from being able to work 40 hours a week on a sustained basis. They further assert that in many instances, the fact that a person can do the tasks listed is often erroneously used against respondents by both DDS and ALJs, particularly in cases where the disabling condition is a mental health one, not a physical or intellectual disability. For these reasons, they suggest removing this section from the form.

SSA Response: When evaluating a CDR, we compare records from the most recent favorable decision (referred to as the Comparison Point Decision (CPD)) to current records to evaluate if medical improvement occurred. This may involve collecting additional information related to their daily activities. During the CDR process, we not only evaluate the prior condition(s) to see if medical improvement occurred, but we may also need to assess any new impairments that developed since the last review. While not every case requires additional information to assess the beneficiary’s daily activities, this section allows the beneficiary to describe their daily life and how their disability affects these activities. We use this information, along with the detailed information provided in this section to determine if we will need to send additional forms to assess how these activities are impacted by their condition(s).

The same commenter also noted that SSA may choose to keep this section in the form. If so, they suggest SSA simply the section by removing open-ended questions, such as Questions 9(A) or 9(B). This commenter believes there is not enough room on the form to include an answer with the level of detail the question asks for and reiterates that the answer is, at best, only indirectly related to the overarching question the CDR is attempting to answer: has the person’s medical condition improved such that they are now able to work. This commenter notes there are also a number of disabilities that would prevent a person from handwriting a sufficient answer. Therefore, this commenter suggests that, if this section remains, the questions should all be yes or no questions with space to provide an optional further explanation, as it is currently formatted, and the instruction should make it clear that further explanation is optional to submit.

SSA Response: We reformatted the section to provide a chart listing each task in the daily activities portion with check boxes to indicate which ones the beneficiary has difficulty in performing. If the beneficiary checks a task, they then provide a brief statement in the field provided. If the respondent needs additional room, they can use the Remarks section. SSA uses this information , along with the specific information provided in this section to determine if we will need to send additional forms to assess how these activities are impacted by their condition(s). In addition, we are also making the two open-ended questions (now questions 8A and 8B) optional for respondents to answer. We are working to remove those questions from the form and screens in the near future.

Another commenter suggests removing Question 9(A) on Page 14 as it is an open‑ended question without much guidance. The commenter notes that asking a individual with a disability what they do in a “typical day” can be difficult for the respondent to answer if they do not have a “typical day.” In addition, the respondent may forget to include pertinent information because the question is open-ended.

SSA Response: This section allows the beneficiary to describe their daily life and how their disability affects their daily activities. SSA uses this information, along with the specific information provided in this section to determine if we will need to send additional forms to assess how these activities are impacted by their condition(s). However, in deference to this comment, and the usability testing we conducted, we are also making this open-ended question (now questions 8A) optional for respondents to answer. We are working to remove this question from the form and screens in the near future.

Finally, while the CCD understands that the information about daily activities, solicited in Section 9, is essential to understanding ongoing disability, they recommend revising this section as follows:
- First, the CCD recommends SSA remove question 9(A), because they note that essay questions are very burdensome for claimants to complete. They also think that question 9(A) is unnecessary because it is duplicative of 9(C).

SSA Response: This section allows the beneficiary to describe their daily life and how their disability affects their daily activities. SSA uses this information, along with the specific information provided in this section to determine if we will need to send additional forms to assess how these activities are impacted by their condition(s). We removed numerous narrative sections throughout other sections of the form to reduce this burden. However, in deference to this comment, and the usability testing we conducted, we are also making this open-ended question (now questions 8A) optional for respondents to answer. We are working to remove this question from the form and screens in the near future.

The CCD recommends keeping the check box responses to “Do you ever have difficulty doing any of the following” in 9(C) but finds them under-inclusive. They recommend adding a box for “Sometimes” to accommodate those claimants who have intermittent problems completing activities of daily living.

The CCD also recommends adding a Question 9(D) which asks, “Do you ever need or get help doing any of the following” and includes the same list of activities, followed by check boxes: Yes, No, or Sometimes. The CCD finds that claimants frequently report they do not have difficulty completing certain tasks, even if they require assistance to do them, if that assistance is already in place (i.e., I don’t difficulty cooking because my partner does all the cooking). To ensure SSA is capturing complete Daily Activity information, the CCD asserts this additional question may be helpful.

SSA Response: We added additional wording if someone checks the YES box to ask: If YES, select any tasks that you need help with or have difficulty doing. We reformatted the section to provide a chart listing each task in the daily activities portion with check boxes to indicate which ones the beneficiary has difficulty in performing. If the respondent checks a task, they would then provide a brief statement in the field provided. If they need additional room, the respondent can use the Remarks section.

Comments Suggesting SSA Complete a Behavioral Audit on the CDR Process and Form

Comment #32: One of the advocacy groups, Ideas42, had the following suggestion for completing a Behavioral Audit and applying Behavioral Design to revise and improve the SSA-454-BK:

Burdens Matter Even More for People Experiencing Scarcity: Ideas42 suggests using tools to address burdens and improve public program design using behavioral science. They cited research they conducted which shows that applying Behavioral Design can address burdens and make it easier for people experiencing scarcity to enroll in and stay on public programs (Daminger et al., 2015; Ideas42, n.d.). Ideas42 notes that the CDR form burdens the public in several areas: cognitive, time, and financial cost demands, and that imposing burdens on people living in poverty (i.e., experiencing chronic scarcity), such as those who have a disability and need financial assistance, can cause disproportionate harm to people experiencing chronic scarcity. They suggest that applying Behavioral Design tools will help address these burdens better for the respondents.

Time Estimates Capture a Narrow Picture of Burden and are Often Inaccurate: Ideas42 suggests reassessing the current time estimates through observing the target populations from start to finish as they gather the necessary materials to answer the questions through submitting the form to SSA and including any necessary follow‑ups. They explain this should include the respondents’ need to take breaks, as well as the need for respondents to request help from others. The commenter also suggests including other types of burden in the estimate, including burden associated with learning costs and psychological costs which are not included but also important for assessing the experience of completing the SSA-454-BK. Finally, the commenter suggests also including the time burden for those potential respondents who never complete the form or fail to start completing it.

Ideas42 notes that administrative burden can be understood as the costs people experience in learning about services (learning costs), complying with the rules, requirements, and paperwork needed to obtain public benefits (compliance costs), and feeling the stress, loss of autonomy, and stigma throughout the process. They state that based on their conversations with partners in the disability community and through their work with other public benefits programs, 60 minutes is likely a gross underestimate of the time for completion of the SSA-454-BK form. They also note that the limitation of the Social Security Administration’s assessment of burden is that it only includes a time burden estimate for applicants for the completion of the form and is too narrow and fails to capture all types of burden. In addition, they believe the current burden estimate also fails to account for the burden placed on others who support the respondent in completing the form. Therefore, Ideas42 believes the Social Security Administration should broaden its consideration of burden when redesigning the SSA-454-BK form to be more inclusive of these types of burden.

Behavioral Audits and Best Practices are Needed to Redesign Successfully: Ideas42 suggests that SSA use behavioral auditing to reassess the SSA-454-BK and redesign the form to: streamline it to reduce the number of questions to only those absolutely necessary and remove duplicative questions; revise the language for clarity and ease of access to make the form more easily navigable, and to explain terms as well as explain why each piece of information is necessary; format the form to place critical information at the top and marking it accordingly; and promote positive identities to avoid stigma associated with signing up for public benefits, or shame in providing the information, rather the questions should be designed to promote dignity and avoid stereotyping. In addition, the commenter suggests engaging with the target population who complete the form to enlist their help in revising the form to better suit their needs while providing SSA with the necessary information.

Ideas42 notes that accurately assessing the full extent of the burden for a form or process requires consideration of all three types of burden: learning costs, compliance costs, and psychological costs. The commenter does this through a process called Behavioral Auditing, which engages mixed methods of research that includes qualitative and quantitative research along with methods such as observation to assess these burden types and suggest opportunities to address them. They suggest the above best practices for conducting a behavioral audit that may assist with the SSA‑454-BK form redesign.

Ideas42 also suggests that SSA should redetermine both accessibility of the form for those who have visual, hearing, motor, and language limitations or other needs. In addition, they suggest providing the form in multiple languages for those who require it. Finally, they recommend a clear and authentic offer of help for the form, which they state should include SSA or DDS staff who can assist with the form and answer questions directly and quickly. They suggest that SSA include clear, accessible contact information on the form that explains that a representative will help the respondent directly. The commenter suggests these practices will help those with disabilities to respond to the form on their own.

Finally, the commenter suggests the Social Security Administration should conduct a more comprehensive assessment of the burden imposed by the SSA‑454‑BK form. They believe that conducting a full audit of these burdens and redesigning the form to account for best practices in behavioral science and human-centered design would be a practical next step, and an important service to those who have a disability and should not face hurdles to accessing needed financial support.

SSA Response #32: We conducted an evaluation of the form and worked with our CDR product discovery workgroup to revise the form by streamlining the questions where appropriate with the goal of reducing the overall burden of completing the form. We appreciate the input on evaluating the form; however, we will not be outsourcing this evaluation.

Comments on the new i454 Internet Modality

Comment #33: One commenter appreciates that SSA is creating a new web-based modality for respondents to complete the CDR form. Providing an opportunity for people to complete the SSA-454-BK form conveniently and with as few burdens as possible is a much-needed step in the right direction. However, SSA should make sure to address all the burdens of completing the new online modality and consider the full scope of the administrative burdens for completing the form to help SSA successfully improve it.

SSA Response #33: Based on the comments we received, SSA is updating the burden estimate for the SSA-454-BK and i454 to an average of 8 hours to read the instruction, gather the data, and complete the form. Since we understand that some commenters report a longer time estimate than 8 hours, we conducted ongoing evaluation of the form and worked with our CDR product discovery workgroup to revise the form by streamlining the questions where appropriate with the goal of reducing the overall burden of completing the form whether it is completed by paper, or the web-based i454.

30-Day Comment Period Federal Register Notice (FRN):

The 30-day Comment Period began on August 18, 2022 and will end on September 19, 2022, at 11:59pm. We will review and respond to any public comments we receive during the 30-day comment period FRN. If we receive any comments in response to this Notice, we will forward them to OMB.

Usability Testing Notes from the Round 1 i454 Sessions Held from 10/18/21 – 10/27/21

Usability Testing with SSI and SSDI Recipients (10/18/21 – 10/27/21):

Sessions from 10/18/21:

12:00pm Session:

The facilitator explained the process to the participant. The facilitator walked the participant through the screens. The participant said she really liked the application and stated the only thing she would change is the extra question asking, “what the type of language do you speak,” as she stated SSA already asked it earlier. One of the note takers from the Usability Experience Group (UXG) asked if people really would want to print the confirmation page, and the participant stated that she believes respondents would want to be able to print the page, in case something happens, and SSA cannot find record of completion. That way the respondent would be able to provide proof of submission.

2:30pm Session: [The participant failed to show, this session was cancelled]

Sessions from 10/19/21:

9:00am Session:

The facilitator explained the process to the participant. The facilitator walked the participant through the screens using the furnished information with ease. The participant said he liked the collection but did not understand why SSA needs to ask for all the information, as he did not think all the information the application requested was relevant. He suggested SSA allow respondents to opt to not input some of the information. He specifically felt SSA should allow respondents to opt out of including the physical information, such as height and weight. He just did not understand why that information is needed, so decided to skip entering it. In addition, in multiple other places, the respondent skipped input where the systems did not require information (e.g., there was no red asterisk reflecting mandatory fields). When asked, he just said that if the section does not have the red asterisk, he thought he could just skip it.

All in all, the participant liked the application, and thought it was easy to use, but he felt it may not be for others (he claimed to be very tech/computer savvy). He thought navigation would be more difficult for respondents who don’t use computers often. In addition, he recommended removing information that is not mandatory to fill out, as that may make it longer to look for respondents to input.

12:00pm Session:

The facilitator explained the process to the participant. The facilitator walked the participant through each screen, and she really liked the ease of the information collection. The participant stated that the application is very easy to use, as long as the respondent reads all of the questions, and as long as they have their doctor’s information, and medications with them when they begin the application. She also stated that she liked how she could see all of the information she had input and make edits, before submitting it.

To summarize, the participant stated she would be comfortable using this application in the future and would recommend it to friends. She stated she would not change anything about the application.

2:30pm Session:

The facilitator explained the process to the participant. The facilitator walked the participant through each screen. The facilitator ensured that the participant understood what she was seeing and could repeat back in her own words the instructions on the screen. The participant felt the screens were pretty straight forward. She thought that the new medical conditions would be on the next page (not on the same page), because that’s how her doctor’s form works, and suggested that it could be confusing for others at first. However, she thought it was very easy to add or delete new condition items, which she liked. She thought it was a bit harder to find the doctor’s information page, as well, but she was able to find it.

The participant stated that she was a bit anxious at the beginning but found it easier to use the application as they went through it. Overall, she thought the design was fine, and she liked the one block that allows respondents to add any additional comments because she said that someone always wants to take away benefits, and this block allows someone to add an explanation or additional information that might help. She felt the screens were mostly straight forward but felt that when it said “add another medical provider” she thought it meant that the first one wasn’t saved and felt that the button there should say “save” instead of “add” to make it easier to understand. She thought it would be pretty easy to use if she needed to use it for the CDR process.

The facilitator asked if the participant had any questions, but the participant had no questions. The facilitator also opened the floor to questions from the observers, but there were no questions from the observers, either.

Note: Kyle Gardiner from OMB was on this call.

Sessions from 10/20/21:

9:00am Session:

The facilitator explained the process to the participant. The participant had a bit of a technical issue with the application, but the UXG team was able to fix it. The facilitator worked through the screens with the participant, as per normal practice.

The participant felt the first screen was pretty easy and went through it very quickly; She also worked through the remaining screens quickly but clicked on everything to see how it works. She explained that she likes to do that when using a new application, as she never knows what something does until she clicks on it. She felt that the medical information was hardest to navigate, and that it would be helpful if SSA had a better explanation for it there. She also had issues with the drop-down boxes, and felt they were not as useful or as all-encompassing as they should be for a CDR.

The participant said that she is concerned that most people might not click on things like she did (as she is always curious about these things), and that it might be helpful if the form told people to click on the drop downs or the other items that may not have actual drop downs. She suggested that the program should offer the respondent the opportunity to save the final report rather than just print it, as not everyone has a printer. Overall, she felt the application works well, but that the instructions need to be clearer. She felt it was easy to navigate. She also felt it might take a long time to gather the information needed to fill out the form, and that some of the questions are worded strangely and they should be defined and clarified better. The participant felt this was easier to use than the paper form, but still felt that it would take the same amount of time to gather the information to fill out either form. Overall, she felt it navigates well, and is easy to follow.

The facilitator asked if the participant had any questions, but she had no questions. The facilitator also opened the floor to questions from the observers. One observer asked the participant to clarify which drop down boxes the participant asked if SSA could improve. The participant stated she would want SSA to improve all of them.

12:00pm Session:

The facilitator explained the process to the participant. The facilitator walked the participant through each screen. This participant navigated through the screens with ease. She really liked the application and didn’t really have any issues with it. The only thing she recommended was to include drop downs or fill ins for the medications and injury/illness blocks, to help with spelling and make the process faster.

All in all, the participant liked the application; thought it was easy to use; and didn’t have any issues. When asked if navigation would be more difficult using a mobile phone, she said it would definitely take longer, but only because of the limited screen view. She would save more often, just to make sure she did not miss something. Also, when asked if much research would be needed prior to being able to fill pout the form, she said she wouldn’t need to, as she would know most of the information off hand.

2:30pm Session:

The participant felt the first screen was very self-explanatory and went through it very quickly; however, the participant felt that there should be definitions for terms (for instance in the Contacts section, he felt SSA should make it clear who the contact should be, if not self; or to use a different term to make it clearer). He also had some issues with the Medical Conditions section and asked lots of questions as to how to fill out the section, as there may be other medical conditions that may not impact the disability directly, but that may still exist. He also asked if these need to be ranked by importance to the disability. He suggested SSA revise the form to include a better explanation for this. He preferred to read every question, even those that did not include a star (i.e., asterisk) in case he did not understand something; however, he stated he may not answer the ones without a star if he were filling it out for himself. The participant had several suggestions for improving the application, especially the remarks section. He suggested SSA should be more detailed as to what respondents should place in each section. He also asked questions about navigating the application.

The participant said the application is very simple to use, but getting the information needed is more difficult. However, he said that it might help if the application stated that it is okay not to have the medical records so long as the respondent gives the medical provider’s name and contact information. He felt the application was clear and concise. He also asked what the cut-off date for using the application would be so that respondents would know to check back by a specific date and not checking daily. In addition, he said it would help if the help screens were available through little blue question marks (like they appear on other forms). He said he would like to use this instead of the paper version and thought it would be helpful to be able to go online earlier to access it and see the system before having to use it.

The facilitator asked if the participant had any questions, but he had no additional questions. The facilitator also opened the floor to questions from the observers, and they had no questions.

Note: Kyle Gardiner from OMB was on this call (he logged on late).

Sessions from 10/21/21:

9:00am Session:

The participant felt that the “information about you” questions can be confusing for people who can’t read, or who can’t speak English. He also had a lot to say about the language choices in the drop down – that someone who doesn’t speak English might not be able to read the English versions of the other languages, so SSA might consider using the name of each language in its native language (not in English). The participant also had some issues in the Contacts section, as he thought it was info about the person completing the form. He said that calling it “Emergency Contact” would make it easier to understand. The participant had some trouble navigating and finding places to add new items. He was also very confused on how to answer the CDR questions in general, and that in most cases, he told the facilitator that, if he does not know the extra details, he leaves it blank rather than putting something in. So, he suggested that they add something to explain why they need it, and what, exactly, SSA needs (e.g., Over the counter meds? Or just prescribed meds? Make it clear) – he also asked if maybe this is something that SSA can get from a doctor, so the respondent might not need to include it? He informed the facilitatory that he does not always remember everything and said that there should be a list at the beginning of the application (in the instructions) of all of the information the respondent needs to answer before they begin the application. The participant mentioned that it would be helpful to have “disclaimer information” to explain that it is okay to say you do not know, but that you may need to give a contact (e.g., a medical provider) who can answer the medical questions, or medication questions, if the claimant doesn’t know them. He also said that he might be afraid to answer some of the questions because the questions ask for things he might not know, or that others might not know. The participant also said that it would be helpful to have a way to contact SSA on each page, because a respondent might need to contact someone to help them while filling out the form. He also reiterated that it would be helpful to have a list of what someone needs to fill out the application before they start so they can gather what they need at the outset, prior to beginning the process of filling out the application. In addition, he said that it might be hard for claimants to fill in the short answer parts, because they might be worried that saying they do nothing all day, for instance, might not help their case. And, that he might be afraid to answer some of the questions on claimant abilities, as he might make a mistake and click that he can do something he can’t do, or that he can’t do something he can, and it might contradict the short answer section, causing problems with the responses on the form, and possibly causing a discontinuation of benefits. In addition, he felt the Remarks section might also cause a respondent to include information that contradicts previous information. He again reiterated that it would be better for SSA to state that a medical specialist could answer all of the questions for the respondent and make it easier for the claimant to feel comfortable answering these questions. He also felt that respondents might doubt themselves and their answers, since SSA doesn’t tell them where to elaborate, and SSA does not explain what SSA needs and why they need it. The participant said he wouldn’t print the receipt because he feels that SSA will just call him with follow up questions, so he might as well not bother to print it. He feels that the receipt just confirms that someone filled out the application, but with all of the “I don’t knows,” SSA will follow up with more questions anyway, and the application is useless. He expects that the follow up will be through several phone calls.

The participant said that the application is complicated, whether it’s online or on paper. He thinks that SSA should really just ask for the doctor’s phone number and not bother with everything else. He doesn’t think he would use the Internet version rather than the paper CDR.

Because the session ran late, the participant logged before the facilitator could finish the session with the evaluation or follow up questions.

Note: The participant for this session has a neurological disorder, and short-term memory issues. He did not mention if he has a Representative Payee, but he likely does have one.

12:00pm Session:

The facilitator explained the process to the participant. The facilitator worked through the screens with the participant, as per the established practice.

The participant stated that the program is easy to understand and to use. Overall, she said she liked the system, and it was easy to use. Her main concern was being kicked out of the online form, as she said she had issues with other SSA forms online timing out and kicking her out and wanted to make sure she would not have the same issue with this one.

In general, though, she said she liked the new system and would be able to use it. However, she did say that she would call in for assistance from the get-go, and not try to complete it on her own. Also, in her opinion, she thinks it would take just as much time to complete as the paper forms, because she would still have to get together all of her information before beginning the process.

2:30pm Session:

The facilitator explained the process to the participant. The facilitator worked through the screens with the participant, as per normal practice.

The participant felt that the first screens were pretty straight forward and legible, and said she would save at the bottom of each page just to be sure it saves. She said the application seems pretty easy to navigate, but she was not happy that the tab button does not work [Note: it only did not work during the usability testing stage, it will work when SSA implements the application]. She said that she was not sure what the application meant by “school” or how adding in a vocational counselor is considered a “school.” She suggested that maybe the form could give more guidance with examples to make it clearer what is expected in that section. The participant said she would be much more detailed if she was completing this for herself. She would try to be thorough but felt that this seems to be a chore as the application causes more and more text boxes to pop up for each yes or no question. She also said that someone might need a larger text box to fill some of these in, if they are being very detailed. Even so, she said that it was clear that if you click “yes” on the ability section, a text box opens for more information. She suggested that SSA might consider adding mental health issues to the list, as well. When the facilitator asked about the Remarks section, the participant said she would add in a summary of the information just to be sure, and to add anything extra that wasn’t covered before. In reviewing the summary section, the participant was concerned that some of the items were not answered, as she did not think that was an option. She said she wouldn’t want to leave anything blank just in case it made a difference. The participant was not happy that the system removes the categories completed list on the right-hand side once the respondent gets to the summary page. However, she was able to find one section in the summary about doctors when prompted by the facilitator even without that list.

The participant said that the application seemed pretty easy, but that it would be better to have the navigation bar to the right at all times, including the summary page. She said that the electronic version seemed easier that the paper CDR, and she was much more worried about the paper version she did recently (whether there was enough room, or whether she forgot to add something). She said she would choose to print the page, and she thought that this would print just a summary of the information filled out, but it might just be the message saying she printed the submission screen. She would want a summary of all the information she filled out and that way, if SSA contacts her, she would have a summary of the information she submitted in front of her.

Overall, the participant thought that most of the application was easy and straight forward to fill out. She thought there would be more room to write things out (more than 500 characters in a text box), and she reiterated that the navigation list of what she had completed should remain on the right. She also said that this would give her a bit more piece of mind that the submitted form would not be lost in the mail. She said her last CDR took her at least a week to complete. She thought that using this online format might be faster and easier, and definitely would be less stressful for submitting it and knowing it went through. Her favorite part was the navigation bar that tracked her progress. She thought she would feel more comfortable doing the CDR this way rather than on paper.

The facilitator asked if she had any questions, and the participant’s only question was about when she would receive payment for the usability session. The facilitator told her she should receive it soon.

The facilitator asked if there were any questions from observers. There were two questions:

One observer asked a follow up on the school section: What kind of schooling did the participant think it asked for?
- The participant thought it meant community college or a customer service program; she did not consider a division of vocational rehabilitation would be classified as a school.
On observer asked about the doctors and medicines section regarding insurance benefits: What kind of insurance benefits did the participant think that meant?
- The participant was thinking that meant health provider insurance only.

The facilitator thanked the participant for her time and let her know that she should receive the payment soon.

Sessions from 10/22/21:

9:00am Session:

The facilitator explained the process to the participant. The participant had some technical difficulties with his slow Internet, which the facilitator tried to work through with him. The facilitator began to work through the screens with the participant, as per normal practice.

As the participant started working through the screens, his Internet connection was not stable, and he dropped from the session. The facilitator tried to trouble shoot with the participant again, but nothing worked, and the participant lost connection again. Because of that, the facilitator needed to end the session early (after half an hour), as we were not able to get through the screens with a faulty Internet connection.

The facilitator thanked the participant for his time, and the participant asked if there might be another time when we could try this. The facilitator said he would discuss with the recruiter and would let the recruiter know if there is another opening for the participant next week.

12:00pm Session:

The facilitator explained the process to the participant. The facilitator worked through the screens with the participant, as per normal practice.

The participant stated that the program is easy to understand and to use, and the only comment she had was she did not understand why we are asking for height without shoes. She continued to go through the questions well and seemed to be understanding the questions and the sections. She said it was easy to navigate and not confusing. She also really liked how easy it is to submit the form. She also said she would print out the receipt confirmation page for her records. She expected that the receipt would just be the page that shows she submitted it and the date submitted. She said it would be useful to print the previous summary page, too, but did not expect to have that information on the final page.

The participant said she thinks that having this Internet version would be good and convenient for anyone who is tech savvy. She liked that she could save the data she entered, and that the application did not lose her information. She also found it convenient that she would not need to mail papers to SSA, as she said that SSA might lose the papers and cause a setback. She also liked that everything corresponds on the website just by logging in. However, she thinks that elderly people might not want to log in and use a computer, so she hopes that SSA continues to use the paper form, too, and will let the respondents choose to use the computer if they want to, or the paper form if they want to. She said it would be helpful for SSA to also send a notice by email to let people know they have a CDR to do and could do it on the computer. Finally, she said that it wasn’t hard to use, but not any faster than the paper form. She felt it would give her peace of mind to use the computer and know the application submitted properly instead of needing to buy extra stamps to hope that the paper form gets to SSA.

The facilitator asked if the participant had any questions, but she had none; the facilitator asked if the observers had any questions, and they had none.

The facilitator thanked the participant for her time and ended the recording.

2:30pm Session:

The facilitator explained the process to the participant. The facilitator worked through the screens with the participant, as per normal practice.

The participant said he liked the application, and thought it was easy to use.

Note: As our observer needed to leave the session early, we did not hear any of this participant’s suggestions for improving the application, or overall thoughts on the application.

Session from 10/25/21:

9:00am Session:

The facilitator explained the process to the participant. The facilitator worked through the screens with the participant, as per normal practice.

The participant had little trouble with the application, taking the lead and asking questions about what he was expected to fill into the form based on the scenario – he was moving ahead of the facilitator, and asked if he needed to fill out things that are not required (with the red asterisk). He let the facilitator know he was hitting save to ensure he would not need to retype anything into the form. He noted that the medical provider section is optional on the form, but he felt that it is not really optional – he asked the facilitator if the respondent does not fill this section in, will SSA would recontact the respondent to obtain that information anyway? The facilitator agreed that is the case, and the respondent felt that the section should either be mandatory, or should inform the respondent that, if they do not fill it in, SSA will contact them. After putting in the medical provider’s phone number, he asked why there was no option to put in a fax number, as the VA normally asks for that. The participant noted a duplicate question (for the language question) and asked why we are asking the question twice, and also said he would not answer either of them, as they are duplicates. For the typical day block, he noted that the information was not required, and that he would not want to fill it out, but he probably would as he thinks that would be one that would cause SSA to recontact him. The facilitator agreed that would likely be the case and gave him a scenario to include in that box. For the Additional Information box, the participant said he would include anything he felt he might not have covered previously in the form. He thought that 6000 characters might not be enough, depending on the type of disability (or disabilities), and how long it has been for the respondent between CDRs. He thought the summary screen was laid out well, and he figured out that he could add items using the edit button – he did not want to explore the form, as he would not want to mess with it. The facilitator asked about the plus icon on the summary screen, and the participant said that he doesn’t like to click on things when he doesn’t know what they do. The facilitator asked the participant to click on the plus to see what it does, then asked how we should explain that – the participant said that the plus and minus is okay, but there needs to be some label that says, “click plus to expand,” as there should be simple instructions so the respondent knows that clicking on it would not undo what they’ve done by hitting a button when they don’t know what it does. On the last screen, the participant said he would save the summary page and he would click print on the last page to save as a PDF and then print it – he thought it would give both a confirmation that he completed and submitted the form, and also that it would give the same summary as on the last past.

The facilitator thanked the participant for his work on the form and asked about overall thoughts. The participant said that it is well organized and easy to go through, and it is better laid out that the VA Internet forms. He thought it seemed “idiot-proof.” He didn’t like it, but thought it was standardized form and not overly complicated, just long and tedious. He thought he would be able to figure it out. And he said he would make the form save automatically whenever someone clicks next, and that it should tell someone it does that. When asked if he had ever filled out the paper version of the form, the participant said, he had, and that filling it out only took about 20 minutes, but gathering the materials took a very long time, as it required constantly finding more information. He thought that the Internet version would have the same problem – where someone needs to gather more and more information as they fill out the form. The facilitator asked if including a list of what is needed to fill out the form would help with that? And the participant said no, as he thought that people would skip it, or ignore it.

The facilitator asked if the participant had any questions, but he had none. The facilitator asked if the observers had questions, and they had several:

One observer asked why the participant thought that 5000 characters might not be enough time to fill out the typical day part.
- The participant said that his whole day is always very highly planned out (300-400 events per day), and he would want to ensure he doesn’t miss anything.
Another observer asked whether the activities listed on the form would be adequate for him.
- The participant said that yes and no, as daily activities are things he might be physically able to do, but not mentally able to do. For instance, his caregiver ensures that he dresses appropriately. He feels that the SSA questions rarely cover everything for those with mental disabilities.
The observer asked a follow up question as to whether a text box would help with that.
- The participant said that it wouldn’t help, as with his mental disability he doesn’t think he would think about these things or think that he is doing anything incorrectly.

The facilitator thanked the participant for his time and thanked him for going through the application so quickly and easily. The participant asked why there was no question about having his caregiver help fill it out – and the facilitator said, he was right, there is no section for that, and that, while a rep payee may have access to the form, we did not have a place to note it. The participant said that he thinks it needs to be on the form, as if a rep payee helps him fill it out it may make it seem that he is much more competent and less disabled than he really is.

Note: this participant has mental/cognitive disabilities and has a rep payee caregiver who helps him.

Session from 10/27/21:

9:00am Session:

The facilitator explained the process to the participant. The facilitator worked through the screens with the participant, as per normal practice.

The participant said he understood the initial questions and was able to work through them quickly, but he prefers to read through everything to ensure the information is all correct. He was questioning the height without shoes and wondering why SSA specifically is asking for height without shoes. He also was not sure why SSA even asks for this data, as it should be the same as it was initially. He said he would still fill it out but does not understand why SSA needs it.

The participant input several items in the section for assistive devices and explained that he has found that it is better to add more there for SSA than to leave it blank or only put in only one thing. He said he was unsure that other respondents would do the same, though. He did not want to fill in the text boxes with the reasons for medication, however, as he said that the government can take things the wrong way, so it is best to give little or no information there.

The participant had no issue with the vocational page, but said he was still not sure why SSA needs it. He also said that the question there about written language use is redundant, as we already asked it earlier when we asked about preferred languages. He told the facilitator he would still answer it, as the government always asks things too many times, or ask for things that don’t seem relevant and you need to give them the answer or it will hold things up, or you might lose your disability benefits. He had no issues filling out the rest of this section.

When the participant got to the section on Daily Activities, he sighed and said he did not really like to fill out things like this. He said he would put in a worst day scenario as disability is based on the person’s worst day, especially if the respondent has more bad days than good ones. He also said he would want to put in everything he could here and felt that the space provided might not be large enough. In the rest of this section, he put in very short answers in the short-answer sections, as he felt after that first one, it was still better to say less than more in the other sections, as he felt that these sections were only there to trip up the respondent into saying something they hadn’t said previously or contradicting their previous information.

In the Additional Information section, the participant said he would include information on why he was initially granted disability, and that he’s still disabled. He would state again the information from before to make sure that the government has it.

On the summary page, he said he would review it for mistakes, and make sure everything is accurate before submitting anything. He would use the edit buttons to edit if needed. He said he liked the final page and would likely print out the page when given the option, but thought it would be better to save it, or to let someone print out the entire report.

In general, he said he doesn’t really like any government form, but felt this one was better than filling out the paper CDR, and he would probably use it, if given the option. He liked being able to type things out rather than having to write everything, as he would rather type than write by hand. He likes the idea of having an Internet option.

The facilitator asked if the participant had any questions, but he had none. The facilitator asked if the observers had questions, and they had several:

One observer mentioned that the participant had typed in some of the dates, and use the drop down for others, she asked how he would prefer to enter that information.
- The participant said he prefers to type it in)

The observer also asked about the patient ID number, as she wanted to know if the participant knows of a way to figure it out.
- The participant said it’s probably a Social Security Number, but he wasn’t sure if it could be something else. He said he would probably leave it blank because of that.

Another observer mentioned that the participant had said he had initially signed up for disability himself and didn’t hire a lawyer, and that he referred to that as a mistake. She asked why he believes it was a mistake.
- The participant said he feels like SSA forces people to hire a lawyer who knows the application process and how to navigate the system, because without a lawyer, people get denied.

That observer also asked about information that the participant said that SSA should already have, for instance he said that SSA should already have the main information (name, address, and English proficiency, etc.), as well as medical information, and she wanted to know why he felt that.
- The participant said that he believes that SSA has this information already through other forms and should be able to send an Internet application like this one with that information already filled out.

The facilitator thanked the participant for rescheduling the session and participating in the testing, then ended the call.

Usability Testing with Advocacy Group Members (10/25/21 – 10/27/22):

Sessions from 10/25/21:

12:00pm Session

Participant: Crysti F., New York State Attorney and Disability Advocate, and a NOSSCR Representative

The facilitator explained the process to the advocate. The facilitator asked the advocate how she usually interacts with the CDR form, and she gave a brief overview of the work she has done working with the disability public to complete the form. The facilitator then started to go through the screens with the advocate, just as the facilitators have done previously with the usability sessions with members of the public.

The advocate stated she is ok with the layout of the first page, she did state that people may not know that the blue language is a hyperlink to start the CDR. The advocate suggested adding language like “click here to complete report.” She mentioned the links on the right under the Overview section and stated they may confuse the user as a respondent might click on and leave the CDR page accidentally. The facilitator informed the advocate that those sections will not be on the final screens.

On the Instruction screen, the advocate asked if the user will receive a re-entry number when they log back on if they save and exit. The facilitator explained that respondents will not receive a re-entry number, but the system will take them back to the last screen they saved.

On the Medical Condition screen, the advocate said she likes the format of having to add each condition separately and stated that it keeps the page clean when adding another medical condition. The facilitator asked the advocate if she thought SSA needs to keep the height and weight section. The advocate said the information might be needed, as it may help determine if the person is obese.

On the Medical Treatment screen, the advocate stated that the question asking if the respondent has an emotional or learning problem should have a longer list of condition descriptions. She gave the following example: Do you suffer from any mental impairments either diagnosed or undiagnosed? She stated that some people may not recognize if they have an emotional or learning problem, as they may have other conditions which are more pronounced.

On the Medical Impairment screens, the advocate stated that SSA may want to make this section clearer as the respondents may not know their diagnosis. The advocate asked if SSA asks whether the respondent has a representative payee anywhere on this application. The facilitator explained that SSA does not expect representative payees to have access to the CDR system, so we did not include that question.

In the Doctor or Healthcare Professional section, the advocate recommended taking out the “home” option from the drop-down list, as the recipient would not have the doctors home number. She recommends adding cell or office number instead.

In the Office Visit section, the advocate asked why SSA asks the respondent to provide information in the last 12 months, but then asks about the First Visit to a healthcare provider. She stated that SSA does not need to know about the first visit, as SSA should already have that information. The advocate said it is more appropriate for SSA to ask, “Last Visit, Next Visit, or how many visits have you had in the last 12 months.”

In the Test Ordered section, the advocate said it is important to collect tests doctors ordered and suggested that SSA could add a dropdown list indicating if the test was performed more than once.

In the Medical Treatment section, the advocate asked why the previous screen took her back to the screen she had just completed. She said she sees no difference in this screen and the previous screen, except it now shows the doctor’s information. The advocate recommended SSA add directions somewhere asking if all doctors have been entered, and maybe providing a link that takes the respondents out of the page if they have included all of the necessary data.

In the Hospital or Clinical section, the advocate stated that SSA should change “first visit” since SSA also states we are asking for data from the last 12 months.

In the Overnight Stay section, the advocate said that some SSI/SSDI recipients may go to the emergency room, and be there all night, and think it is an overnight visit, even though they have not been admitted. She suggested SSA clarify that this section is for admitted hospital stays. She also recommended placing “Hospital Stay 1 (Date In/Date Out)” etc., side by side, as the recipient may look at these as two different hospital stays.

In the Add Hospital or Clinic section, the advocate suggested taking out the doctor information and just listing the name of the hospital.

In the Medicines section, the advocate suggested keeping all the information on the screen, instead of bouncing back and forth to different screens.

For the Other Medical Information section, the advocate stated that the section is pretty descriptive, but in the Medical Organization section suggested that SSA change this section to “Add Other Source,” as the respondent may get confused if they have a long-term or short-term carrier. She also suggested SSA change the dates in this section to Date of first Contact within the last 12 months. In addition, she suggests that SSA provide a description as to what SSA is looking for in the Reason for Contact box.

In the Education and Training section, the advocate said the claimant may not remember when their last disability decision was and suggested that SSA can auto populate the date. She also mentioned that if SSA wants all of this information, SSA should not limit the section to say “school.”

In the Daily Activities section, the advocate suggested that the section should be more descriptive, for instance adding, “since you stopped working, what is your typical day like for you?”

In the Hobby or Interest section, the advocate made it clear that she does not like this section and suggested that SSA needs to request the number of hobbies or interests which the respondents engage, if they do so at all. She states this section as it stands is open to interpretation.

In the Taking Medicines section, the advocate suggests that this section may need more description including adding a section on remembering to take medicine.

In the Daily Activities section, the advocate recommended that SSA change this to “Daily Activities (Currently Involved In),” and said this question should be asked during an interview on the phone as the recipient will likely require significant guidance.

In the Completing Tasks section, she suggests that SSA needs to be more specific and explain what SSA is asking here, as otherwise respondents will not be able to fill it out.

On the Remarks screen, the advocate recommended that SSA add “Additional information that will help us understand your condition, or some language that allows the recipient to provide additional information they believe SSA does not have.” She believes this will clarify this screen.

On the Receipt screen, the advocate asked if it was possible for SSA to auto populate the recipients local field office on the summary page, as she believes it would be helpful.

The advocate completed the assessment but was not satisfied with the information asked on the website, and believes the website was hard to understand, and would be difficult for most of her clients.

As no one had any follow up questions, the facilitator thanked the advocate for participating in the testing, then ended the call.

2:30pm Session:

Participant: Josh D., California Attorney (Veterans’ Disability advocate), Disability Rights California, The Arc of Sacramento

Note: This session started late as the advocate was late to join (he joined 15 minutes late), because he did not realize that the session was on East Coast and not West Coast time.

The facilitator explained the process to the advocate. The facilitator asked the advocate how he usually interacts with the CDR form, and the advocate gave a brief overview of the work he has done working with the disability public to complete the form. He also said that he is disabled himself, so he is very familiar with the process, and often uses Dragon voice-recognition software to complete the forms.

When the facilitator questioned this, the advocate said he needs to use voice‑recognition software, as he is a quadriplegic, and is unable to fill out forms without it, and the facilitator said that Dragon software might not work with the testing software, but we will try it and see. The facilitator informed the advocate that SSA plans to look at the prototype to see if we can make it compliant with voice recognition software, and that SSA understand the need to have 508 compliant applications.

The facilitator then started to go through the screens with the advocate (reading them for the advocate in case the voice recognition software did not work). The advocate did not have much to say, other than his voice recognition software does not work with the test screens. He asked the purpose of the red star (asterisk) but was able to find that answer for himself. And said that, so far these are all questions that are on the normal CDR form. He was not very talkative, and the facilitator asked him if he could speak his thoughts out loud to let us know what he’s thinking about the screens.

The advocate asked if the application could ask for multiple contacts in the contact section. The facilitator said that the paper form only asks for one, so the Internet application also only asks for one. The advocate said that there could be multiple contacts/caregivers, and it might be better to include more than one contact here. He also suggested that adding “caregiver” and “employer” in the contact categories drop‑down, as that might be useful.

The advocate asked if there is a specific order in which respondents need to list their impairments/conditions for the CDR, and he noted that often respondents will list the ones that are bothering them most, but not all of them. He suggested that the form should tell people to list them in a specific order, and to list all of their conditions and not just some of them. He recommended rephrasing the question to “what impairments are you obtaining benefits for?”

Once the advocate tried it, he had no issue using voice recognition to fill in the section and found it very fast and easy to do. Although, he said that there needs to be more information on what the respondent needs to give here, for instance the underlying conditions and the symptoms. He also said it would be helpful if SSA made it clear as to whether any weight is given to other symptoms, or just to underlying conditions.

He liked that the height and weight was in both SI and metric units, as that could be helpful for immigrants. Although he said that the question is not really necessary unless height is relevant to the condition, and it could trip some respondents up who might put in the height and weight differently each time (reducing accuracy or causing a discrepancy). He also thought that better examples of other assistive devices or technology would be helpful as an explanation of what SSA is looking for here (for instance, someone may have multiple assistive devices, like this advocate has), and the respondent needs to know how many devices they should list, so SSA should make it clearer in this section.

For the healthcare information, the advocate thought it would be more useful for SSA to collect data from the last CDR completed, and not just request the information for the past 12 months. He said a CDR should be more about reviewing changes since the last medical review, and less about asking for the same things SSA requested previously. He also suggested that SSA ask the respondent for individuals who might have that information to provide to SSA, rather than asking it of the respondent. He also suggested removing the doctor’s home phone as it is not necessary, since no one calls their doctor at home.

For the doctors’ appointment section, the advocate was concerned about the visit dates, and said that last visit might be the most important – especially for someone who is reevaluated often, and he was not sure why this was optional data. He also said that SSA should ask for the dates of more important office visits rather than just first and last, as that would require the doctor to write up more information on that appointment. Although, he said SSA could leave it as is, just to see if the person is receiving regular care. However, he also suggested that SSA could ask “how frequently do you visit your doctor?” which would also resolve this issue he has with this section of the form.

The advocate agreed with asking the questions about adding a hospital or clinic. He was also satisfied with the medications listing but asked if there was a field asking if someone is still taking the medication and suggested that would be an important thing to note. He had no other issues with the medication section and liked that the list of medications updated as he added more medications to it.

The facilitator asked if the advocate agreed with SSA’s asking questions about the vocational information and education section. The advocate said that these questions were relevant as they ask about Ticket to Work. He had no real issues with the vocational/education page, but felt it really only reflects those who are in the Ticket to Work program.

In the work section, the advocate had some questions about why SSA is asking about work since the last disability decision instead of “in the past 12 months,” as it differs in consistency from the rest of the form. He had no major comments on the rest of the application, other than it seems to follow the paper CDR consistently.

The advocate talked a bit more about how he helps people fill out their CDR forms, and about obtaining and gathering the information needed so there are no delays in processing the CDR. The facilitator asked him how long it takes to process and complete the CDR, and the advocate said that it can take several days or a few weeks to gather all of the data and complete the form. The advocate also stated that if someone was treated at certain facilities it can take even longer to get the medical information, and those with mental illness may have more difficulty getting accurate information or remembering information. However, this advocate said it takes an average of three weeks to complete everything, and his clients would not be able to do these things by themselves. The facilitator asked how long it takes to fill out the form once the advocate has all the data, and the advocate said about 6-7 hours, as the medical history might be fairly quick, but the daily activities can take a very long time, and with his assistive technology, it can take longer to fill out any form. The advocate pointed out that a 500-word essays takes a while to complete, and the form requires several of them to provide an adequate picture of a person’s daily activities and current medical improvement (or lack of improvement).

The facilitator asked if the observers had any questions, and there were none. The facilitator thanked the advocate for participating in the testing, then ended the call.

Note: We had two unknown Observers during this call (possibly from another advocacy agency, or from OMB).

This advocate is also a quadriplegic and required voice recognition software to fill in the form.

Sessions from 10/26/21:

9:00am Session:

Participant: Richard W., Attorney and Disability Advocate, Community Legal Services of Philadelphia, NOSSCR representative.

We waited 25 minutes for the advocate, but he did not show up, so the UXG team needed to call this session as a no-show. They said they would inform ORRC and OMB if they choose to reschedule this advocate for another time.

12:00pm Session:

Participant: Alan P., New Jersey Attorney and Disability Advocate, NOSSCR representative.

The facilitator started the meeting by explaining what we are doing and asking for the advocate’s background.

The advocate said he is an attorney, and he started at SSA as a lawyer for what is now OHO, and worked for an ALJ, writing decisions (back in the 1970’s), then he left SSA and started working as a lawyer and Representative Payee to represent claimants at both state and federal level, and he also helps with CDRs in his capacity as a Representative Payee and Appointed Representative.

The facilitator then explained how we have been using the new i454 screens and asked the advocate to follow along and go through it with us to give his opinions on the new screens and their utility.

The advocate said that he completes online applications all the time, and asked why we are restricting this application to just the claimants, and not others who help them or assist them? The facilitator explained that for right now this will be just for those who have mySSA accounts, so first person respondents only. The advocate said that most of his clients may not have access to computers, or may have a phone only, or may need assistance to complete the online form and will not be able to do the application on their own. He suggested again that this should be accessible to the representative payees to assist.

In reviewing the first screens, the advocate suggested that we should also consider including the last 12-month stipulation on other forms as well, like the hearings forms and appeals forms. He also noted that in the Contacts section, someone might not have a contact, and suggested that we add a box stating, “I don’t have a contact.” The advocate also suggested that SSA should have the representative listed in the system, and that should appear on the form, if the claimant has a representative. He suggested that we add that in, and have the system retrieve it from SSA’s systems, since this application should only be accessible from a mySSA account.

In the Medical Conditions section, the advocate suggested that the explanation of this section should tell the respondent to list all medical conditions, and any other conditions (medical or non-medical) that might affect them, not just the ones that limit their ability to work. He said that adding that stipulation might help respondents with their claim and would also clarify what SSA needs here.

The advocate also recommended that SSA should only request respondents list the assistive devices that meet the medical listings, and not bother with the others. He said SSA should clarify what they are looking for there (not just give examples), by stating that respondents should include any device they might use to assist them. He suggested a drop-down box may be more helpful here.

For the medical provider section, the advocate suggested also adding in nurse practitioners, and PAs to the list, maybe MSWs as well, as the drop down will need to include the other types of medical providers they may see that are also included in SSA’s regulations. He also suggested that SSA should only ask for the doctor’s work number, or not worry about the phone type as that’s just their number. He also suggested that claimants never know their patient ID number, and SSA might as well not ask for it, as it is probably left blank most of the time.

The advocate said that when he fills out the short-answer items for the Medical Conditions, he does not bother to fill out much information there, as SSA probably wants to hear from the doctor anyway, so he doesn’t see that it is useful to put in much information in those boxes. He suggested that most respondent likely include information such as, “treats me for ___” or something short. He said that these are not so useful for claimants to fill out anyway, as the medical provider and evidence should speak for themselves.

In the section regarding treatments and medical testing, the advocate said he would list each treatment in the box and give a reason for each treatment and test done, and he said that takes a vast amount of time. He asked why SSA really needs this information when SSA is requesting the records anyway, and the claimant could be wrong in how they present the information. He felt this is also an unnecessary box.

For the Doctor and Healthcare Professionals section, he asked why SSA needs this if we already asked for the medical providers. He asked, “what more information could SSA possibly need respondents to include?” He also said this section may be useful if someone forgot to add a doctor, but, otherwise, it isn’t needed, and he suggested SSA remove it.

For the vocational section, the advocate said that Ticket to Work is rare in his opinion, and the others should either be in SSA’s system, or really have nothing to do with a CDR or potential cessation of payments. He suggested that SSA should just ask for vocational rehabilitation sessions, and, if there is one, to note that SSA would need to ask the respondent for more information later.

For the Daily Activities section, the advocate did not believe that the information about a typical day would be useful, as the state agencies (DDSs) actually ask the same thing in a form of their own. He feels this is also not going to give SSA useful information, as it could vary too much from claimant to claimant. He also expressed concern that judges might use that information against the claimants, while the claimant may not even remember putting that information on a form. He also stated that the open-ended format is a bit better here, but still not particularly useful. He said that these sections could be more harmful to people than good, as they may not explain enough, or might explain too much, and may not include information that will aid SSA in adjudicating the case. He suggested that this information is not necessary, especially if SSA is going to rely on the information from the doctors. He also expressed concern that some people will not even answer these questions or talk about the problems they have in their daily lives, or they may not know, as they may have caregivers who normally relay this information for them. Regardless, he reiterated that this section should not be necessary if SSA is asking for medical information from the medical providers.

In the Additional information section, the advocate thought the provided text box might be large enough, but said he knows that on appeals it is possible to attach other documents, and that might be useful here, too. The facilitator said that we don’t do that now for CDRs, which is why we did not include it on the Internet version of the form.

The advocate felt that the summary screen was useful, and should give an option to print or save, instead of just offering the option to print. He liked that the summary screen gives an opportunity to edit. However, he reiterated that this would be a good place for someone to upload attachments with more information. He suggested again that uploaded attachments would be more valuable than the short-answer text boxes. In general, he felt most of his clients would not print the screen, as they would expect it to be a receipt only, and not a full summary of the information they submitted. He said representative payees would likely print the screen for their files, though, but felt that SSA should send the receipt and summary to the respondent via email instead, as that would be more helpful. And, if that is not something SSA could do, then at least send a link to the mySSA account where SSA should be able to show that information.

The advocate said that he believes this new application is fine, but he has concerns about SSA’s Internet applications in general. He said it would be more helpful for representative payees and appointed representatives to be able to submit the CDR online, and not just first-person users. He also said that it is an issue that respondents and their representatives are not able to submit an appeal for a CDR denial, which is a huge problem. He also said that respondents or their representatives should be able to request a continuance online as well. He expressed concern that the more people who have a hand in these SSA forms, the more issues crop up. He also expressed a complaint about the SSA-1696 and asked how the new submittable version gets sent forward and feels that there are more problems with it now as it needs to go back and forth between representatives and claimants, which is an issue. He also reiterated that it would be more helpful for representatives to file a CDR online and be able to file an appeal on a CDR decision online, too. He said that dealing with the paper versions just slows down the process too much.

Finally, he suggested that SSA should inform doctors that they will hear from SSA directly, and to remind claimants to stay in touch with their doctors for that reason. He also stated that SSA is delusional if they think that claimants will fill out this application on their own. While the claimants may be the ones to log into their mySSA account and submit the documentation, the likelihood is that they will ask their representatives or other advocates to help them complete the CDR screens. He did not believe that the Internet version would significantly change the time estimates for completing the form. He said it takes at least an hour to complete, but only after the claimant gathers all of the information needed, which can take weeks. He suggested, again, that SSA could shorten this form by just asking for the medical provider’s clinic, as most people go to a clinic and not a specific doctor, or SSA could do what the DDSs do, and just ask for the name and phone number of the doctor, since that is much easier for the claimant or their representative to find.

The facilitator asked if the observers had any questions, and there were none. The facilitator thanked the advocate for participating in the testing, then ended the call.

2:30pm Session:

Participant: James H., Attorney, Bay Area Legal Aid (Oakland, CA)

The facilitator started the meeting by explaining what we are doing and asking for the advocate’s background.

The advocate introduced himself and discussed his experience in Social Security matters. He stated that he has worked with clients on Social Security issues for three years. He conveyed that he has a good working knowledge of Title XVI claims, with some experience in Title II claims.

As the usability session began, the facilitator discussed the application at a high level. The UXG group displayed the landing page, but the facilitator stated it would change “quite a bit” in the final iteration of it that is ultimately incorporated behind the mySSA portal. The advocate asked and was told how to access the application.

The advocate opined that SSA should not make the question about providing a contact name of someone else who could talk to the claimant’s disability mandatory. He said that not every claimant has someone close enough to them to do so. He also explained that many times, claimants may inadvertently give the name of a contact who would be unhelpful to their claim (because the person might actually not know all the details of the impairment and how it affects them). The facilitator informed the advocate that this information was essential because the DDSs need someone to contact if they cannot reach the beneficiary.

In turn, the advocate said the iCDR application opening screens reminded him very much of the reconsideration form. He pointed out that a contact name is optional there, and so he thought it should be optional for the iCDR as well.

The advocate “did not like” the fields labelled “Medical Condition 1” and “Medical Condition 2.” He stated they are problematic because a CDR is evaluating if disability benefits should continue, and frequently disability is granted on a complex mix of factors, including a combination of conditions and how they interact with each other. Separating out medical conditions into discrete, separate categories would obscure this complexity.

He asserted that asking for height and weight information is “funny,” and that he did not understand why SSA asked for this information.

The advocate said the application itself was mostly fine. He said his clients probably could not complete the application due to their impairments, but that was the case even for the paper version. He would be able to help his clients through it, but he doubted that most people with a disability without representation could complete the iCDR (or CDR in general).

There is a field in the application that makes it optional for the claimant to provide the name/contact information for a hospital or clinic that could provide more information about their disability. The advocate wants these fields to be mandatory because these sources can provide important current records SSA might not have.

The advocate also expressed that the question “Has this doctor ordered any tests?” would probably only produce a “yes” or “no,” but that most clients would not remember the specifics. The facilitator pointed out there is actually a drop-down menu with test names to select, and the advocate said that was probably helpful.

The advocate conveyed concern about the questions regarding a first versus last office visit. He stated, “there could be important ones in the middle,” and that the respondents would need a longitudinal measure to properly convey the trajectory of their disability. He also said the questions about doctor visits should go further back than 2 years, since, for example, a homeless claimant would probably not have seen a doctor during that time.

In lieu of the question asking, “What treatment was received,” the advocate prefers more specific questions such as ones asking about “medication management” and “psychotherapy received.”

He asked why the application was asking about over the counter medications and expressed concern that SSA would make value judgements based on them.

He said he did not understand why the application asks about which written language claimants use, since it did not seem relevant to him.

Regarding the question “what year did you last attend any school,” the advocate asked that we clarify which type of school we were referring to (college? Primary school?).

The advocate opined that the estimated formal public reporting burden for the current CDR form, and thus presumably the iCDR application, was too low. He said that for most of his clients, it took at least 90 minutes to complete the CDR form with his assistance.

In closing, the advocate expressed that his overall reaction to the idea of an online CDR was positive: “Anything going online and away from paper is great.” However, he asked if he could discuss some concerns with the CDR process in general. These included the following statements:

The CDR process is repetitive, challenging, and frequently seems almost pointless. Consistently, the advocate said his experience was that he and a client would complete a CDR, but then the local DDS would “ignore it altogether” and send them a “function form” with questions the DDS had created on their own, many of which duplicated the CDR. Accordingly, the advocate came to view the CDR as a “foot in the door,” to initiate the process. Knowing he will need to duplicate much of the information, he does not spend too much making sure the CDR is complete and just waits to hear from the DDS. He frequently avoids answering time-consuming questions, “since the real work is with the DDS anyway.”

He also stated that colleagues in other states experience the same thing, and that the DDS questionnaires they receive are not the same as is his. He said he appreciates each state has its own practices, but that he wishes for “more uniformity and less redundancy.”

The advocate also said he experienced this with other forms besides the CDR (DDS coming up with their own versions).

Continuing in this vein, the advocate stated he does not understand who is reading the ultimate data and is actually making the decision about whether disability payments should continue.

The facilitator asked if the observers had any questions, and there were none. The facilitator thanked the advocate for participating in the testing, then ended the call.

Sessions from 10/27/21:

12:00pm Session:

Participant: Sacha M., California Attorney and Disability Advocate.

The facilitator started the meeting by explaining what we are doing and asking for the advocate’s background.

The advocate stated that she is relatively new to doing CDRs, and has only helped one person so far, but she is familiar with working with the disability forms and the population of people with a disability as an advocate and appointed representative.

The facilitator then explained how we have been using the new i454 screens, and let the advocate know again that we are looking for her feedback. The facilitator asked the advocate to follow along and go through the screens with us to give her feedback wherever she has comments and sees places for improvement. Then the facilitator asked if the advocate has any questions before we began, and the advocate did not.

The facilitator asked the advocate to go through the first screen to see if she had any questions or concerns. The advocate said it looks good to her. The facilitator asked if there are any issues with the terms we used, like CDR, or Continuing Disability Report, and the advocate said that her clients may not call it by that name, as they might call it a reevaluation or a review, but not CDR; however, she did not think that her clients would use this title, she would use plain English instead, for instance “review of disability.” She also stated that she does not believe her clients would understand the language on the screen and felt it would be better if it was phased using plain language.

The facilitator had the advocate click on the CDR link and look through the first page. She asked for the advocate’s comments. The advocate said the first section of the instructions are a bit confusing and clunky, but it is okay enough for someone to figure out. The facilitator asked what the instructions should contain, and the advocate suggested it should just say that the form will contain “information about yourself, information about your medical treatment, etc.” The advocate said that column with the list on the side of each section is useful, but most of her clients would probably ignore it, or not notice it. The advocate also said that for the CDR, many of her clients ask her why SSA needs this information, as SSA should already have it. They want to know why SSA is asking it. For instance, SSA should already have their doctors on file, and suggested that SSA should only ask for new doctors instead of the ones currently on file. The facilitator asked what the advocate would see as the ideal situation. The advocate said that the form should be tailored to the claimant (for instance with pre-filled info), as that would be less tedious, and it would show that SSA isn’t requiring them to repeat the whole process over again.

The advocate also said that the placement of the buttons at the bottom of the page are not intuitive. She said the Next button is on the wrong side, as she feels it should be where the Save and Exit button is. She suggested that this is also not intuitive and not helpful for her clients.

Looking at the first page (the Information About You section), the advocate said that many of her clients are mono-lingual and don’t speak English, and it is very difficult for them to fill out these forms. She said it would be clearer if we provided these forms in other languages. Also, she said it would be helpful if choosing a language from the drop-down box would then switch the whole form to that language. Although she noted that the language drop down does not include all languages there, just some, and it could be misleading if someone selects “other” and can’t get an interpreter because of that. She believes SSA should mention that on the form as well.

In the Information About You section, the advocate also said that she wanted to know why SSA asks the questions in this manner [the manner in which we ask them on the CDR and the iCDR screens]. She said that some of these are ambiguous, as respondents do not know what SSA wants to know when the agency asks things like “can you write more than your name in English.” She wanted to know if this question is asking if they can write down all their thoughts in English, or just some words in English, because the question is not clear. Regardless, she believes it would be helpful for SSA to explain why they need this information.

The advocate also questioned the other names section. She said some people are transitioned and might not want to share their dead name [birth name], and SSA might not accept their chosen name. She feels this is an issue SSA needs to address to be more inclusive.

The Advocate had comments on the Contact section, as well. She said it is clearly not an option to fill this out, but she said many people get upset and don’t want to include a contact name for various reasons, or don’t believe they have a contact who can really help them with this information. She also wanted to know why SSA needs to know the specifics of who that contact is to the respondent (like father or stepfather, or friend, or other). She said it was fine but believes it should not be required. She also felt that some of the Contact information seems to be intrusive and not necessary to ask. She said that SSA should already have Contact names from older submissions, and it would be better for SSA to pre-fill this part and then ask if the respondent has other contacts to add, or to delete as needed.

In the Medical Conditions section, the advocate said that she likes the part where you can add new conditions, said it’s easier than using the paper form. However, she said it might be easier if SSA would allow for the medical conditions to be in one box, instead of separating them. The advocate suggested it would also make sense to ask progressively additional questions here. For instance, if someone adds a medical condition, SSA should then ask “what medications are you taking for it? Which doctor are you seeing for it?” Then move to the next condition. She also said she would suggest having the height and weight questions first, then the conditions, as she prefers to handle the medical questions last. Even then, she questioned why SSA even needs this information [about height, weight, and medical conditions]. She asked why this is important to ask, if SSA is asking for medical providers anyway.

The advocate also had comments on the assistive device question. She felt it was confusing as written, as it asks always/sometimes/never before the box, and, as someone may have multiple devices that they use at different times, it would be more helpful to ask those questions for each device (and not for all of the devices as a whole). She also felt that the list of example devices is okay, as they seem to show up most often, but some people might not consider glasses or contact lenses as an assistive device. She believes SSA needs to make this section clearer with better instructions.

In the Medical Treatment section, the advocate suggested that SSA shouldn’t use the word “problems” under mental conditions, as that is SSA referring to mental conditions as a problem. Instead, SSA should just call these conditions, or issues. She also felt that this section should be tailored to the person completing the form, to add new medical records and not have to resubmit the old ones. In addition, she felt this section should say “since the last time you completed a CDR or Disability decision” and not use the last 12 months. She also felt that SSA should already have this information and shouldn’t need to ask for it again. She suggested again that SSA should pre-fill this section and then ask if the current information has changed. She also said that clients might not always know the name of the doctor they saw. They may remember the place and time but not the doctor.

For the Doctors and Health Care Professionals section, the advocate reiterated that someone doing this on their own might not know the name or contact information for the doctor. However, she felt the respondents may know the facility in which they were treated. As for the visit dates, the advocate believes that a text field where you can type in the date is better than the drop-down boxes. She also said that some people might not remember dates as much as times of year/seasons, so this information may not be accurate, and it would be better for SSA to just contact the medical facility to ask for documentation. She also recommended that SSA not ask for the first visit, as it could have been years ago, and may not be something the claimant can remember.

The advocate believes that the Medical Conditions Treated section is also very confusing. She pointed out that some treatments include medication, but the question asks specifically for treatments that do not include medication. She also said that some people do not consider a doctor visit to be “treatment.” The advocate mentioned that she considers therapy, counselling, medications, or surgery as treatments, but not an evaluation during a doctor’s appointment. She also suggested making some changes to the wording in this section. For instance, she would change the word “counselling” to therapy, as that is closer to the term used in the medical listings. The advocate also reiterated that she feels the Medical Condition questions should all be linear, once the respondent mentions one, then the system should ask about the doctors seen for that condition, medications taken, treatments/tests for it, etc. She believes a more linear method would make more sense logically, and then SSA would not need to ask questions that seem to be asking for the same thing over again.

For the tests section, she also said that the drop-down box is not conclusive and felt that the “other” option is likely what many respondents will choose. She also said that people often don’t remember when the test happened by date, they are more likely to recall a month or time of year. In addition, she pointed out that many respondents may not know what the doctor was testing for, or even what the test was called, making this section even more confusing. In addition, she pointed out that SSA will get the entirety of this medical documentation, including dates for tests and emergency room visits, when they request the information from the medical provider. She did not think SSA needs to ask for this information from the claimant when they will receive it later anyway.

For the Medicines section, the advocate likes that the medications prepopulated on the screen from the information previously provided. However, she said the instructions were not clear as to whether the claimant should add in all medications (like over the counter pain killers), or just the prescribed medications, or even just those medications taken over long periods. She suggested that SSA clarify this in the instructions. She liked the drop-down box that allows the claimant to include the “prescribed by” information.

For the Other Medical Information section, the advocate asked if the next button automatically saves inputs. If so, she asked why SSA has a save button at all. The facilitator said that they have heard other comments about that, as well, and are considering these comments.

For the Education and Other Training section, the advocate questioned the types of organizations SSA lists there as examples. She believes the list is a bit confusing, as medical organizations are not vocational, and neither are prisons. She said SSA needs to be clearer on what they want here, and maybe just state “other organizations” that have records. She also said that it’s confusing to ask for “Date of first contact” and then add “within the last 12 months” as it should really ask “how often do you see this organization?” The advocate reiterated that the way this is phrased is confusing. She also asked how SSA expects someone to input the information if a claimant went to school or training, but dropped out? Finally, she did not believe the last three questions on the screen connected well with the first two. She thinks they should be with the other language questions earlier in application.

In the vocational section, the advocate said that the question about school is confusing, as it seems to be about asking if someone is getting school in relation to getting jobs, but it is not clear. The advocate said that IEPs and work programs are very different. She said that SSA would likely want records from any of the programs. She also mentioned that the question “what year did you last attend school?” seems unrelated to the vocational items, so it might be better for SSA to rephrase this section. The advocate also felt that this section had too many bulleted points, and they are also confusing. She reiterated that this section should also be pre-populated depending on the options the respondent chose on the previous screens or based on current information SSA has in the respondent’s records.

For the Daily Activity section, the advocate said that these questions are very uncomfortable for most people to answer. She said her clients find it traumatizing to have to write down that they can’t do much, and in sections like this one, they don’t know how much to say or how much to write. She said it’s hard to tell them what to focus on, and how to share their activities. She pointed out that some people do not really have typical day. She reiterated that these questions under are always uncomfortable questions to ask and noted that the CDR includes a very long list of them. She feels these questions are intrusive, and not really necessary to ask. She also asked about the 500 characters, and wondered how long it really is; however, more importantly, she felt that asking for a statement for each of these items might make people feel the need to repeat themselves or add more information than needed.

For the Work section, the advocate asked why SSA needs to collect all this information, and why this section isn’t in the vocational section. She feels this is asking the same things again and again, and said it feels confusing as to why SSA can’t just contact others for more information instead of having to ask these questions again.

For the Remarks/Additional Information section, the advocate felt that people might share things from their personal experience in this section, but as a lawyer and representative payee, she said representatives tend to not fill this section out as the clients are too tired by then. She also felt that most people may skip this section or use it to express their feelings for how long the overall process takes.

On the Summary page, she asked that if something is incomplete will it allow for submission? The facilitator said that there would be a yellow mark, not a green one, and it would not allow submission until the required information is added. The advocate felt that was fine, but not fair for people who may not have that information to add.

On the Confirmation page, the advocate said she would print and save the receipt. She said she would expect the receipt to have the summary in it, if not the whole form.

The facilitator asked for the advocate for her overall comments. The advocate responded that she felt that her clients might not be able to do this on their own as it asks for too much information and is too involved for most of them. She believes the application asks too many questions and does not feel that respondents will fill it out completely. She did not have anything that really stuck out as something she liked about the application. She felt the phrasing of questions are uncomfortable and invalidating, and she feels like SSA is asking for this information too often. The advocate also reiterated that she feels SSA asks the same questions multiple times on the application, and that SSA should really only ask for updates instead of requiring claimants to fill out the entire application with the same information SSA already has on file.

The facilitator asked how SSA might change the application to make it better. The advocate said the form should really be tailored to the respondent, and to requesting updates instead of requiring respondents to rehash all this information which SSA should already have on file. Also, she suggested the application have fewer buttons to press, and fewer questions to answer, so that respondents do not need to click through so many things, and so the form will not feel as impersonal. She suggested this might minimize the trauma caused by having to answer these questions in such a cold way.

The facilitator asked how the advocate completed the CDR with her last client. The advocate said she filled out what they could initially, then had to do some research to fill out the rest of the form, from requesting records and then updating information on the form that might be missing.

The facilitator asked about the function report sent by the DDS, and the advocate said that that function report asks a lot of the same questions that the CDR asks, and it’s also long, but not as long as the CDR. The advocate said that the DDS form is also bad, and she feels like SSA should already have this information since the DDS is already asking for it. Plus, she said the work history report also asks these questions, and it feels like SSA is just asking for the same things too many times. She also stated that she believes SSA is trying to trip people up with these questions.

The facilitator then asked if any observers have questions, and they asked the following questions:

One observer asked the advocate how long it took to fill out the CDR
- The advocate said it took about 2 hours.

Another observer followed up and asked if the initial SSI application takes the same amount of time or longer.
- The advocate said it takes about the same amount of time.

The facilitator asked if the observers had any questions, and there were none. The facilitator thanked the advocate for participating in the testing, then ended the call.

2:30pm Session:

Participant: Steven W., California Attorney and Disability Advocate, Bay Area Legal Aid (Oakland, CA).

The advocate for this session cancelled. The UXG team said they would inform ORRC and OMB if they choose to reschedule this advocate for another time.

Round 1 Usability Testing Overall SSA Evaluation:

Background:

The Medical CDR Product’s goal is to modernize the Continuing Disability Review process to provide an online service option to the public and reduce the burden on the field office for processing medical CDRs. The minimum viable product (MVP) is to offer an online service delivery option to complete the SSA-454. Recently, the agency received feedback that criticized certain aspects of the SSA-454 form. To gather first-hand feedback from disability recipients regarding these concerns, Usability Testing was conducted.

Methodology:

There were 19 usability testing sessions. There were 14 sessions with beneficiaries and 5 sessions with advocates. The sessions were completed virtually. During each session, the participants were given a scenario and asked to complete different sections of the form while the facilitator asked probing questions.

Key Findings:

There were multiple recurring issues. Note these issues are consistent with previously obtained feedback.

Mistrust of the SSA and its processes
Non-required information is used as an excuse to not provide information
Duplicative data entry within Medical CDR and with other agency forms
The same type of data in multiple areas within CDR
The organization/grouping of questions is confusing
Unclear instructions
Why add so much detail if SSA will contact medical providers anyway?
No time saved using electronic form versus paper form because all the work is in gathering the records

A deeper dive was completed for areas of concern previously identified. The researchers also took a closer look at parts of the application that are unique to the online service delivery option. For example, the landing page, instructions, receipt, and summary.

Conclusion:

Although there were multiple areas of concern identified, participants held a generally positive attitude toward the Medical CDR application. However, there is room for improvement. The results of the satisfaction survey indicate:

Satisfaction Survey Questions	Disagree	Agree
I thought this website felt like an official SSA website.		100%
I thought this website made it hard for me to complete the medical report.	77%
I thought it was easy to use this website.		100%
I am not satisfied with the information provided on this website.	77%
I thought it was easy to move through the different sections of this website.	92%
I felt it was hard to understand the terms used on this website.	77%
I am satisfied with the speed at which I can complete the report with this website.	92%
I thought it was difficult to find what I needed on this website.	85%
I think I could quickly learn to use this website.		92%
I felt unsure about using this website.	92%

Usability Testing Notes from the Round 2 i454 Sessions Held from 3/29/22 – 4/8/22

Usability Testing with Advocacy Group Members (3/29/22 – 4/1/22):

Session from 3/29/22:

9:00am Session

Participant: Stacy C., NOSSCR

The facilitator explained that these screens are on a new platform, and that, while it looks fully formatted, we are still looking at a prototype which SSA may still need to revise. We expect the participants to review the screens, and if they want to explore something that is not active, yet, we can discuss it. The facilitator then explained the process to the advocate and let her know that we are looking for feedback, especially candid feedback so we can improve the screens. Then asked if they have verbal consent to record the session, and if the advocate has any questions before starting.

The facilitator then said that we are working with a scenario here, that you received, and asked how the advocate works with the CDR process – she said that she works as one of the head administrators as NOSSCR, and her goal is to make it as easy as possible for the advocates to work with SSA and to interact with SSA, and part of that is making the new CDR process as easy as possible for folks.

The facilitator then started to go through the screens with the advocate. The advocate said she would start by clicking on the CDR block from the mySSA screen, and that she felt it was pretty clear, but could have been made more obvious. The facilitator agreed that it could be more visible and thanked her for the feedback. Then asked the advocate to review the information and give feedback on the Medical Release form (SSA-827). She said that it is clear and easy to follow, but that it may still be too long for some people. And then she clicked on the SSA-827 and noted that it is not completed, yet. The advocate asked if the form would auto-fill, or if this is something SSA would fill out later after the respondent clicks that they agree to eSign the release. The facilitator asked if that is something that the advocate thought should be filled out automatically? And the advocate said that the SSA-827 is not the easiest form to navigate, and it is also not the online version here, but the paper form which is more difficult to use and very small to read. Also suggested that we might rename it “release form” instead of “medical release form.” And, that is would be better if it either automatically filled out, or if it would state that SSA will fill it out for the respondent.

The facilitator asked if they could go back to the medical release page – and asked what would happen if we decline as a response, and it popped up a warning, and the advocate noted that declining to sign the release form also means that they can’t complete the CDR online, which is sort of unfair, and also requires the respondent to fill out two paper forms instead of completing everything online.

The advocate said that this is one of the issues NOSSCR has with iAppeals, that if the respondent doesn’t fill out both the 827 and the iAppeals at the same time then SSA does not allow the use of iAppeals, and this looks like the same thing, which is unfair, as there should be some way for SSA to allow them to match up later. Rather than talking about iAppeals more, the advocate said she would just click that she agreed to sign so we could move on with the new iCDR.

That brought them to a new Getting Ready for Submission screen, and the advocate said that it might be better to have this info as to what people need even more up front (before signing anything) – since now someone who is doing a CDR would need to go and find all of these items, which can take time. However, she thought it was good that the list says that they don’t need their medical records so long as they signed the release, but that some beneficiaries might see this as a strange thing, as they might be used to needing their medical records to fill out the CDR.

The advocate also asked if the time zones listed in the help section are different depending on where someone is who is filling it out, or if those are all eastern time zones, and it might be helpful to mention that.

Moving on to the Information About You page, and the advocate asked about the questions regarding speaking and understanding English, and liked that there is a drop-down box, but said that it might be more helpful up front to see if someone is helping them complete the form (especially if they don’t understand or read English, and the screens seem to be in English). The facilitator said that, while we can’t know if someone is helping them, if someone is in through mySSA, then we expect they are filling it out on their own. The Advocate asked if SSA would send this form to someone who is not English proficient and SSA knows they can’t communicate in English (e.g., someone who already receives notices in Spanish), and, if so, then SSA shouldn’t hold it against that person if they choose to fill out the form online (likely with help), and then SSA stops sending them notices in the language they need. Also, if they say they can’t understand English, does the system let them keep filling out the form? If so, then how does SSA know if they fill it out correctly? Maybe the question here should be “do you need an interpreter?” Also, she noted that SSA just changed the rules regarding inability to communicate in English and now the people who fill out CDRs are more vulnerable, and she is worried as to how SSA might treat those people depending on how they answer these questions here. The facilitator asked about the questions at the bottom of this screen (about reading and writing), and the advocate said that she thinks that most people don’t learn how to read or write in English from one CDR to the next (normally), but she thinks that this is just another way for SSA to place people in the rolls, and she has reservations about the usefulness of these questions in general (both on the paper and the electronic form).

Then they moved to the Someone We Can Contact page – advocate was happy to see the drop-down list included an advocate and a rep, as well as other. As for the contact’s name, the advocate said that not everything is a person’s name (like care facility or hospital) and said it might be more helpful to have relationship first, and then change the name depending on what the respondent uses, since the contact’s name might not be a person (it might be the facility). The advocate agreed that the rest of the page looked fine.

Then they moved to the Medical Conditions page. The advocate noted that you still need to put in at least one medical condition and noted that someone can put in more. The facilitator asked if the advocate could put in some conditions just to have them here. The advocate said that she liked that the height and weight is both in SI and metric system. The facilitator asked if she knows why we ask those – the advocate said that it is for calculating BMI as obesity is an issue. The advocate also said that the assistive devices list looks good but is not all inclusive. She also said that SSA might want to add things like “do you use the cane with one or two hands or use one crutch or two” – also said that cochlear implant should be part of hearing aid. Also, said that maybe there should be a checkbox for one cane or two canes, or wheelchair/power scooter, or maybe it should flag for the DDSs to ask for more info here. However, what is simpler for the person filling it out is not always enough info for the DDSs, and maybe ask what the DDSs need given the new medical listings. The advocate noted that for “other” it pops up a new box to specify, which is good.

They moved to the Medical Providers page. The advocate liked that it pre-prepares the information from the last review. The advocate asked if it is a first CDR, would it pre-populate from the initial application? And the facilitator said, yes. The advocate felt this was useful and likes that it allows someone to change details as needed or add/remove providers. She also liked the “I don’t remember” option for the last time someone has seen the provider. The advocate said that she assumed SSA needs this info to request medical records, and that is why we ask these questions/ ask for updates. She did want to know why SSA needs to know when the respondent last saw the provider, if we are also saying “within the last 12 months” – maybe just say “please confirm if you saw this provider in the last 12 months,” and if SSA doesn’t need this info, then don’t ask it. The facilitator asked what the advocate would do if they haven’t seen a provider in the past 12 months, and she said she would delete it – and liked that a popup box came up asking if she was sure she wanted to delete. The facilitator also asked what the advocate would do if needed to add a new provider, and the went through the process to add a new one. The facilitator asked if the status column (reviewed, needs review, new) is useful, and the advocate said that was good. She also said that it is good that we are not asking what treatment they received from each provider.

Then they moved to Tests screen. The Advocate chose to add in a test, and liked the drop-down box, but asked why HIV test was separate from blood test. The facilitator said that all of these come from the form, and the advocate said that it looks pretty good, and allows for an “other” category to add another test if it is not on the list. She liked that the drop-down box for “ordered by” includes an “I don’t know” option. The advocate said that she assumed that SSA needs this information to see where to get medical records (for instance, if the primary care provider asked for a test, will it be in the records from the provider, or will that come from somewhere else). The facilitator said that might not be the case, but it does help SSA track this data for obtaining medical records.

They then moved to the Medicines screens. The advocate liked that it auto‑propagates with the data on file. She found it interesting that adding a new one also gives a box for someone to give a reason why they are taking the meds, but that a review does not include that data, or a space for it.

At this point, the advocate asked if she could click on “save and exit” to see what happens. And suggested that it should give data on how to reenter the form as needed. And the facilitator said it tells them their data is saved and then brings them back to the main mySSA page. The advocate said that it should tell them the info is saved, and how to get back in, and to remind them of the due date for the CDR, so the respondent doesn’t forget it. The advocate said that the medicines page is where people might drop out, so this a good place to talk about save and exit, and a good place for people to get back in.

They moved to Other Medical Info page, and the advocate thought this was fine, but noted that it did not state on the medical provider page to ask if someone is scheduled to see another doctor, and that might be useful before this screen.

Then they moved on to Organization Details screens. Said maybe not to include contact person here, or a box that said they don’t have one, as they might not have one for an organization. She also felt that the Reason for Contact text box might not be so useful, as someone might put in a name there, and maybe a drop-down box would be more useful. It also means that someone could put down anything in that box and it might not be useful information. Maybe just ask “who else might have medical information for you?” and “what kind of information might they have?” but this looks like something people don’t use much.

Moving to the Education and Training page, the advocate suggested this seems like something most people don’t need, but that it seems fine otherwise. She tried adding a new education submission and said that it all makes sense. Said it might be useful to ask if someone is in a special ed program or needs accommodations. And said that it might be helpful to see if someone is in a program but need accommodations.

Then they moved to Support Services screen, and the advocate said that this one made sense and seems easy enough. She tried adding a new support service and found it easy to add/navigate. Checked to see that the links on the screen worked to take people to PASS and to ticket to work.

Then they moved to Daily Activities screen. The advocate felt that the typical day text box is daunting, but felt it is important. Said that for the hobbies and interests box, it might be useful to make it clear the respondent should fill it out if they have help with the hobby, and to focus on how the medical conditions affect their ability to do the hobby. That would better explain how their disability affects them. It also might be helpful to ask how they’ve changes (as in if there are any recent changes in their abilities). She liked the check boxes, and liked the text box below to explain, but said that it might be useful to add something that says that someone doesn’t have difficulty doing these things. Also, pointed out that the CDR form requires people to do most of these things that are on the checklist.

After that, they moved to Work screen. The advocate found a typo on the screen. However, she liked that the first screen was short, but noted that it does not ask any further question about the work. She asked if that would trigger a work CDR? The facilitator said, yes, that would require a different type of form which is separate from the i454. The advocate felt that a rep might think that they are missing something, but it is definitely faster no to need to include it; however, she suggested adding in language stating that “if you’re working, we may need to contact you to discuss it further.

They moved to the Remarks screen, and the advocate said it looks good, but had nothing to add. Therefore, they moved on to summary screen. The advocate asked if, in real life, clicking on Edit will take you to completed page, and the facilitator responded that yes, it will. The advocate said it was good, and useful, but only good for those who can read and understand it.

Then they moved to Submit button. The advocate liked that a popup came up making sure the respondent wants to submit it. On the next screen, the advocate said that it might be useful for it to say that someone could print that page (the one that summaries that it was submitted), although she liked that you can click on the links to see the completed report and signed medical release. Said that it might be useful to have something at the beginning on the medical release stating that the respondent will have a chance to print the form at the end. Also, she asked if there was a reason to have it be separate, as most people will want to have both. The facilitator thanked her for the suggestion but asked her to click on the SSA-827 to see what happens, and the advocate noticed that it was the form completed. Even so, the advocate said that it would be better to allow for printing of both at the same time. She also said that “Done” is a bit confusing, and suggested that the button say, “return to mySSA.”

The advocate said that on the mySSA page it might be useful to say that the CDR was submitted, and no more action required, and then have a link to get back to the item to print, if someone forgot to print. The facilitator said that would be useful and thanked her for the suggestion.

The facilitator said she had a couple question now that we finished:

Thanked the advocate for the feedback – asked for overall thoughts:
- Advocate said it is a hard form for people to fill out, but understands why SSA needs the info to do a medical CDR
Is there anything the advocate would add regarding how to make the form better?
- Advocate said maybe add something that asks what help people receive in completing the form (just because we know people will likely have help completing it)
As far as being notified that someone is due to complete the form, how should SSA notify people?
- Advocate said that as many ways as possible: getting a letter; making sure that notices go to the correct place, escalating over time (maybe a paper letter and message through mySSA), maybe a phone call or a letter to the person who they indicated previously may have more info/contact person. Since bad things happen when they don’t fill out the form, it’s important to ensure that there are multiple levels of outreach
- Also, if you know they speak a lang other than English, then send the notice in that language; use special benefits notices
How long do you think it takes to fill out a CDR?
- A very long time, because SSA is asking for a lot of information, and it is a fair amount of time to complete it.
- She also said that typing in conditions can be difficult for people, as can updating the info.
Asked if the advocate thinks that asking for the medicines info and providers is important, if SSA is just going to get it anyway
- The advocate said if it will help SSA continue benefits, but if it is not an issue, then skipping it will save time
Asked if it takes a long time if someone needs to complete it with a rep?
- The advocate said that, if they have all of their info, maybe not as long, but without it, it can take weeks.

The facilitator asked for the satisfaction survey for the advocate to complete. Then asked for any other questions/comments. There were none.

NOTE: This session went over by 10 minutes. We had no observers from OMB. Naomi did let the advocate know that this session (and the other usability testing) will be part of the official PRA record.

Sessions from 3/30/22:

12:00pm Session

Participant: Richard W., attorney, legal service program in Philadelphia, PA

The meeting began at 12:17 p.m. as we were waiting on the participant.

Prior to the usability session’s formal beginning, the advocate introduced himself, and stated that he works for a legal service program in Philadelphia, PA, where he represents claimants, and his specialty is CDR’s mostly for cases where there are questions in improvement and for people turning 18 years old.

The facilitator began the usability session by displaying the Welcome screen, and explaining we are not testing this page, but we need to access to get into the application. The facilitator stated this page would change “quite a bit” in the final iteration of that is ultimately incorporated behind the mySSA portal. The advocate stated that the term “CDR” was foreign to other uses and said it may be better if we changed the title to “Your Disability Case.”

The facilitator then told the advocate how to access the application.

They moved on to the Medical Release form page. The advocate stated he thought it was strange to see medical release form, and then immediately jumped to discuss “Things we need,” and then continued to the medical release information at the bottom of the page, he, stated we should move the “things we need” to the bottom of the page. The facilitator was not sure if the advocate was seeing the same screens as he would if he were on a computer, as he was on his tablet. The advocate logged into his home computer and used that instead of his tablet; he stated the screens did not look any different from his tablet view.

They moved on to the agree or decline electronically section on the Medical Release Form page. The advocate stated the decline button option “I refuse to electronically sign or want to” language may not ring a bell with everyone. The advocate clicked on the decline button, and stated it took him a little longer to know what they were asking, where it says, “both electronically or both by paper.” The advocate stated that people do not always have a printer, or their printer does not work, and said we should have something to allow them to see the page electronically right now or use the paper version.

The facilitator stated we are looking to get away from paper, but we are still sending packages, and they can use the forms that are mailed to them, or request SSA to send them the form if they did not receive it.

The advocate stated that the mail system is terrible right now, especially in the lower income neighborhoods, and sometimes it takes a month for him to get mail from SSA. The advocate also said that some people are more comfortable with paper, and SSA may want to work some scheduling in, or add an option for them to call SSA to get a copy in the mail.

Then they moved on to the I agree button on the Medical Release Form page, the advocate suggested we might want to change it to state, “to change your mind to electrically sign select above.”

Then they moved on to the Instruction Page/Information About Vocational Rehabilitation section, the advocate stated that people may not know what this means, and parents may not know, or the dates they have could have been when they were younger. The advocate suggested that SSA could add a date section or leave wide open to capture. The facilitator responded that they might be able to add it to the dates field.

They then moved on to the Information About You page. On the "can you speak or understand English section, the advocate said he finds it interesting that we ask this question and suggested that people are going to need help on this page, depending on how well a person reads English; they might ask someone in their house to help them. The facilitator stated that SSA is only offering English and said that if a person can’t read or write English, they probably would not be on the site. The advocate agreed.

Then they moved on to the Information About You/Written language page. The advocate stated that several of these questions are problematic: 1) do you use every day in most situations (homework, school); 2) Can you read a simple message in the language you identified above? 3) Can you write a simple message in the language you identified above? The advocate suggested that SSA may want to shorten the questions, and drop “you identified above,” as that states people are more likely to determine the difference between read and write.

They moved on to the Someone We can Contact page. The advocate stated that this is an important question. For the Relationship dropdown section, the advocate suggested SSA may want to remove legal aid (nonprofit organization) as clients don’t know what this means. He also said we should add social worker or case manager as an option, because some respondents may have a case worker.

Then they moved on to the Medical Condition page/What assistive devises section. The advocate stated he finds it odd that the arrangement of walking device is separate from walker and wheelchair. The facilitator responded that the devices are listed in alphabetical order.

From there, they moved to the Height section on the Medical Condition page. The advocate stated that it seems funny to ask for height. He said he is a senior citizen so the last time his height was measured he was told his height decreased some. The advocate suggested this question is relevant only for children but said that the weight question is a good question to ask.

Then they moved on to the Medical Provider Section on the Medical Condition page. The advocate suggested that people may not be sure when the last time they saw their provider was and said we may want to delete “within the last 12 months.” The facilitator informed the advocate that the DDS only request information for the last 12 months. The advocate suggested that maybe we should change it to “on or about the last 12 months.” He also said that the title “Review and update medical providers” seems a little fancy, and suggests we change it to “Tell us about your medical providers. In addition, for the “When did you see this provider in the last 12 months” section, the advocate said a lot of people will click “I do not remember,” he stated that they may remember the doctor they see all of the time, but may not know the other dates. The advocate also said that people with disabilities do not have reliable calendars to check the dates. He suggested we may want to add something like “Has the Dr. office changed addresses, or “Do you see another Dr. at the same place?” He stated that his suggested questions would encourage respondents to make the changes. The advocate also suggested we may want to change the options “Needs Review” to “Please review” because respondents may not know they are the ones who need to review the information.

Due to time constraints the facilitator skipped to the Daily Activates Section.

On the Daily Activities page, the advocate stated he would like to see the first box broken into two portions: Describe what a typical day is; and has your medical condition changed your daily activity. He said that having them all as the same section becomes an issue where the respondent needs to write an essay.

They moved on to the Do you have difficulty in Doing any of the Following section, and the advocate noted that the list is long, and it may be more comfortable to the user if it was broken down into groups. He suggested we should talk about the quality of what the respondents are doing, because asking questions like “prepare meal” may cause the respondent to say they can fix a bowl of cereal and think of this as making a meal. The advocate suggested that “Driving or using public transportation” should defiantly be separated, because there is a big difference between the two.

Due to time constraints, the facilitator skipped to the end, and asked what for the advocate’s thoughts on the Medical CDR, or at least on what he has seen. The advocate said there are no questions about how the respondents are doing (improvement questions) and felt that a person with cognitive issues may have learned how to make a peanut butter and jelly sandwich but that doesn’t mean they are able to be a discussion to change a decision. In addition, the advocate stated it would be fairer if some questions ask if things changed much since the last review. H also said his paralegals help review cases before he sends them, but suggested we need to talk about quality when it comes to tasks (for instance, “Can you only do simple things, etc.”).

The facilitator then asked the advocate how long his people spend helping clients complete forms. The advocate responded that it depends on the client, but can take between 1 to 2 hours, depending on how much information his client has on hand, but they may need 1-2 sessions, because they may need a different house member to provide additional information

The facilitator stated the object of this new Internet version is to reduce burden (test, medications), and asked if the advocate felt it had. The advocate said that the questions about the doctor, hospital, name of clinic and their phone numbers slows people down. He stated that state agencies have the hospital numbers and thinks that question should not be mandatory. He suggested that SSA should add something to let the respondents know if they do not know they can skip the questions.

General Closing Comments:

This session went over the scheduled time, and the advocate had to leave. The advocate’s overall reaction to the screens he reviewed on the online CDR was positive, and he had a few suggestions (as listed above). The advocate thanked SSA and stated that he appreciates us working on this. He also reiterated that SSA needs to ask more questions regarding the quality of respondents.

2:30pm Session:

Participant: Joanne P., of Bay Area Legal Aid, CA.

The facilitator explained that these screens are on a new platform, and that, while it looks fully formatted, we are still looking at a prototype which SSA may still need to revise. The facilitator then explained the process to the advocate and let her know that we are looking for feedback, especially candid feedback so we can improve the screens, and that we will record the session and take notes but will never include her name in our reports. Then asked if the advocate gives verbal consent to record the session, and if the advocate has any questions before starting.

The facilitator then said that we are working with a basic scenario here with John Smith receiving a letter that the CDR is due and can be completed online.

Before starting the process, the facilitator asked how the advocate works with the CDR process. The advocate said she works for Bay Area Legal Aid in CA, to work with disabled individuals and with SSA on CDRs after an initial notice of cessation. She said most of the people they work with are homeless, or part of vulnerable populations, most of whom don’t have regular access to computers. She said there are issues with getting online for most of her clients.

The facilitator then started to go through the screens with the advocate. The advocate said she liked the red “Needs Action” label and felt that the CDR link was easy to find.

Then they looked at the Medical Release page. The advocate said that it should be more explicit that signing the release form will give SSA the right to ask for medical records, but otherwise, looks good. The facilitator asked the advocate to look at the buttons at the bottom of the screen and asked what she thinks here – the advocate said that the top one looks like it would sign the form and the 2^nd would decline to sign. The facilitator asked the advocate to click on “decline” and take a moment to read the screen. The advocate said “so, if you decline, you can’t continue to use this tool?” She said it differs from iAppeals that way, which might be a bit confusing, as iAppeals gives an option to complete things in multiple ways, but the language is clear. The facilitator then asked to switch it to “I agree” so they could continue with the session.

They moved to the Instructions page: the facilitator asked the advocate to read through and let us know if she has any questions or comments. She said it looks good.

Then they moved to the Information About You page, and the facilitator asked the advocate to review the page and give feedback. The advocate asked if the tool is available in other languages? If not, it should say so. Also, that the page should say that SSA will provide an interpreter earlier on in the process. The facilitator asked the advocate to look at the last question before we move forward. The advocate said it looks good.

Moved to the Someone We Can Contact page. The advocate said it looks okay, but she said that this is a question that needs more context – maybe giving an example (case manager, family member, friend), and a brief message as to how SSA will use this information (e.g., getting in touch with the respondent, or asking for more info). The advocate felt that, if this person will be contacted about medical issues, then it could change how the respondent will respond. The facilitator asked the advocate to look at the drop down. And the advocate said that “case manager” should be added because it’s a common one. Also, said that saying “non-profit organization/legal aid” is confusing and should be separated as they are not the same thing.

They moved to the Medical Conditions page. The facilitator asked about the height and weight questions. The advocate had no comments on height and weight, but noted that there was no star there, so that should mean that we don’t need to input that information, right? The facilitator said she was unsure, but it should be optional. The facilitator asked about the last portion of the page. The advocate thought the list looks good and liked that there is an Other option. There was nothing there that she thought SSA missed.

Then they moved to the Medical Providers page. The facilitator asked the advocate to review the instructions below the question. The advocate said that SSA should include “case management,” as some respondents may consider that their medical provider. The facilitator asked her to click “yes” to review the new information. The advocate asked where this information was pulled from. The facilitator asked her to read the info on the page for that info. The advocate said then this is a 2^nd CDR? The facilitator said it could be from initial disability report. The advocate liked that; however, she said that SSA might want to change “last review” to “the last time you provided this medical provider information” – to be less confusing. The facilitator asked if the advocate noted the “Needs Review” tags – asked if those are clear. The advocate said, then it looks like we need to review and click on the blue link. The facilitator asked her to try it. The advocate did and said “okay.” The facilitator asked about the date the respondent last saw the provider – the advocate said that she likes it better than the questions on the paper form, as that requires more info, but she wants to know how SSA uses that info, and why they need to ask it. If someone is saying that they saw this provider in the last 12 months, then why do we need this info at all? The facilitator said that it is useful for the DDS to have, but they will request data for the past 12 months regardless. The advocate said then they shouldn’t ask for it and should just ask for records for as far back as practicable, and not just 12 months, and not based on any date the respondent last saw the provider, as they might not even know that info, and they just guess. Although, this is a huge improvement over the paper form, but still not something SSA should ask. She also appreciated the “I don’t remember” box.

They returned to the Medical Providers page – and deleted the one entry. They also looked at adding a new entry, and the advocate said it looks good. The advocate asked if reviewing the records is necessary – the facilitator said no, but we want to encourage the respondents to review/delete these as needed.

Then they moved to Tests screen. The facilitator asked the advocate to click on “add test” and look at the drop down. The advocate said it looks good. Then she looked at the “ordered by” list and she liked that list, too. She also liked that it no longer asks for a date for the tests. She had no other comments on this page.

They moved to the Medicines screens. The facilitator asked if the advocate had any comments on the 12 months here. The advocate said, yes, because she would ask for all records (not just 12 months), although newer meds are more important that older ones. She also felt that medical records are a better snapshot than medications. The facilitator asked how many meds do people need to document typically? And the advocate said within the 3-9 range, and that the meds may change a lot, especially psych meds. The facilitator asked the advocate to choose “yes.” The advocate liked that it pulls up a list of known meds for the respondent. The facilitator asked her to click “add,” and review the info requested. The advocate said it looks good, but that there used to be a section for side-effects. The facilitator said the current CDR does not include that – the advocate said it might be a different form (function report) that she was thinking about. However, she thinks side-effects might be useful here. She also said that not everyone will remember who prescribed the meds to them, or the reason they need them, but they will definitely know the side-effects of the meds. The advocate liked that this page was set up like the Medical Provider page.

Moved to Other Medical Info page. The advocate said that case management maybe should be here, too as an example. The facilitator asked her to click on “yes” and “add organization” to see if it all makes sense. The advocate said she has the same comment here about the date – that it should not be relevant, and would be based on an inaccurate guess, also that claim number might not be known, and name of provider might also be confusing – maybe it should just say “name of organization.” She liked the flexibility of “reason for contact.”

Then they moved to the Education and Training page. The advocate said that adding in from the last disability date is useful. Then they selected “yes” and “add education.” The facilitator asked if anything sticks out for her – the advocate said that she is more comfortable with dates in this context, as the respondents are more likely to have accurate info for these. She thinks the question “when did you complete or scheduled to complete” seems confusing and suggested that maybe it should say “or are scheduled to complete.” She said her clients who go back to school often do so with an IEP – is there a section for that, too? They moved back to main screen and looked at the 2^nd question, then clicked “Add.” The advocate had the same comment on the “scheduled to complete” section. She also said there should be a larger text box on this page.

Next, they moved to Support Services screen. The advocate asked if this page only shows up if the respondent adds education. The facilitator said “no, everyone sees it.” The advocate said she finds it a bit confusing because the question asks about services to go to work, and the respondents might not think about IEP or as something that helps them work – they might see it more as something related to education. She said it should be included under education. The facilitator asked about the links to TTW and PASS and said that we had this as a suggestion from the DDSs and the regions. The advocate said that looks good. The facilitator asked the advocate to click “add” and look at the next screen. The advocate liked it, but again, said that IEP or accommodations at school as more about education than work. The facilitator asked what the advocate thinks about the dates here. The advocate said it is less burdensome here, but that she still leans toward asking for fewer dates in general. The facilitator asked about the last check list, and said this used to be a text box, but we turned it into a check list of examples. The advocate said she thinks it’s okay, but isn’t a comprehensive list, especially if we are lumping in IEPs, as those have more testing/evaluations. Again, she said that there should be a question about the sorts of accommodations the respondent has for going to school, as the type of support varies for that. However, she likes the “other” category, as it makes it possible to add something.

They moved to Daily Activities screen. The advocate did not like the question about hobbies or interests, as she said it is a problematic question that is often cherry picked from the form to make a decision for a cessation, and she feels this is a demeaning question and very problematic. She said how well or how often the respondents can engage in hobbies or interests is more important, and that this question is really only used in problematic ways to cease benefits, and never in the favor of the beneficiaries. The facilitator asked about the text box about the daily activities. The advocate said that it is better than the hobbies, but it is still problematic, as it depends on how the respondent phrases things and the information may not be reliable, and this question is often used in problematic ways by the ALJ (who will rely on this to reject the rest of the evidence showing severe impairment). She said that the only way the info on this page is used is harmful for the respondent. The facilitator asked if this information is useful, especially given that we have a function report. The Advocate said, no, this should not be necessary info, and it’s concerning that this is necessary information to provide. The facilitator asked the advocate to look at the long list at the bottom. The advocate said that it’s a very long list, and is already onerous to review, and she never sees it used in any way other than to cease benefits, or in ways that are harmful to the beneficiary. The facilitator asked if someone would be better off not filling out this section, and the advocate said that, yes, it would be better if the respondent does not complete this page at all, as the info here is never used in a way that is beneficial to the respondent. She added that these are decisions that her agency regularly gets overturned because the ALJ relied on this info.

They then moved to Work screen. The facilitator aske the advocate to select “yes” and review the screen. The advocate said it makes sense. The advocate asked, “if someone says they are working, will that info be captured somewhere else?” The facilitator said, yes, the FO will send a form out if someone answers yes.

Next, they moved to the Remarks screen. The advocate said it looks fine.

They moved to summary screen. The facilitator said that tends to be long and asked the advocate to look at the blue text box at the top of the page regarding the signed medical release. The advocate said she would suggest that SSA makes it pop out more – maybe in bold, or red text, or some other way to remind people and flag it again. The facilitator asked the advocate to scroll down and comment. The advocate said it looks good.

They next moved to the Submit button. The advocate liked the pop up in green box saying that the info was successfully submitted. The facilitator asked if the respondent would print or save the review. The advocate said that, yes, an advocate would definitely save the info, and would recommend the respondent do the same. The facilitator asked what the advocate thinks of the “done” button. The advocate said it looks fine.

The advocate said that on the mySSA page she likes that the page shows that the information was submitted and offers a PDF of it.

The facilitator said she had a couple question now that we finished:

The facilitator first thanked the advocate for the feedback, and asked for overall thoughts:

The advocate said that she likes that SSA got rid of a lot of dates and the medical info, and that would be great. She also suggested again that we get rid of the Daily Activities page, too. She said the only other suggestion is that we also allow for use from the SSA.gov main page and allow advocates to help others fill it out.

The facilitator asked: as far as being notified that someone is due to complete the form, how should SSA notify people?

The advocate said SSA should notify people in as many ways as possible.

The facilitator asked: how long do you think it takes to fill out a CDR?

The advocate said that they typically get involved when a cessation has been proposed, so they are more likely to complete an appeal, and that takes about an hour. However, CDRs usually take a long time. She likes that this one pulls information from previous reports, as that is helpful in making it faster.

The facilitator asked if the advocate has any questions. The advocate said no, and thanked SSA for the opportunity to review this form.

The facilitator thanked the advocate and ended the session.

NOTE: The facilitator did not do the survey for this one, as it was going late. We had no observers from OMB.

Session from 3/31/22:

9:00am Session

Participant: Crysti F., attorney, of Private Practice from NY

The advocate started the session with a significant concern that the claimant will receive a request from OMB without notifying their rep payee, or legal advisor – especially as they likely need help with the screens, but their advocate might not be helping them. She was adamant that this not happen.

The facilitator then said that we are working with a basic scenario here with John Smith receiving a letter that the CDR is due and can be completed online. The facilitator let the advocate know that SSA will still send a letter, but that letter will give the claimant the option to use the screens instead of the paper form.

The facilitator then started to go through the screens with the advocate and asked her to review the first page. The advocate mentioned that they have overpayment details here and said that could be helpful. The facilitator asked her to move back to the top of the screen for the CDR process. The advocate still looked at the rest of the screen and asked if she could click on those other items. The facilitator answered “no, only what we need for CDR.” The advocate tried to click on “Action Required,” which did not work, but then clicked above it, and said it would be helpful to make it clearer where to click.

Then they looked at the Medical Release page. The advocate said it looked fine. The facilitator asked her to click on “I decline” and asked for feedback. The advocate said it looks clear and is helpful that it tells the claimant they can still apply but must do so via paper. The facilitator asked that we click on agree now to move forward. The advocate liked how that worked, too.

Then they moved to the Gathering Information page and the facilitator asked the advocate to look over. The advocate mentioned that the contact info should not be a medical contact, and it should be clear that it should be a friend or family contact, not a doctor.

Then they moved to the Information About You page, and the facilitator asked the advocate to look over, and said she can place in there what she wants, as it will not save anything. The advocate asked about the drop-down box for languages; she wanted to know if the screens will convert to the language chosen. The facilitator let her know it doesn’t. Then she wanted to know if the respondent clicked to say that they don’t know English, will it drop them out of the screens? And the answer was “no.” She felt that it should be converted to the language of choice, or stop them from going forward, and since the answer is that it doesn’t do either, the advocate said that is not helpful, as it means anything further will be unreliable information (since it’s not in a language the claimant knows/understands).

They then moved to the Someone We Can Contact page. Advocate liked this page and said it looks like what she’s used to. She only included answers that are required. The advocate said she would continue as if she was a Spanish speaker with no English. She said she understands why SSA asks the question about languages but feels that this form should be available in other languages. She also wanted to know why some of the things on this page are not required – she said height and weight should be required, as should be assistive devices (and SSA should include an option for “does not apply”), as that would be important if someone did not include a device they have because it wasn’t required. The advocate also suggested that for Canes we should include whether they use one or two canes and whether they are right-handed or left-handed canes. She said that if they check canes, maybe SSA could add these cane options to a drop-down list to ask how many and which hand.

Then they moved to the Medical Conditions page. The advocate read through and basically skipped it.

The advocate then moved to the Medical Providers page. The advocate felt this page was pretty straight forward. She thought the initial question was useful. The facilitator asked the advocate to click “yes.” The advocate liked that it brings up a list, and then clicked the first one for review. The advocate was worried about the “when did you last see the provider,” but said that it was already in the system, so she was not worried about when the claimant began seeing the provider, but otherwise liked the review screen.

They then returned to the Medical Providers page. The advocate also liked that the main screen had updated to show that the first one was reviewed. The facilitator asked the advocate to look at the message in blue. The advocate said she would want this page to be clearer that this is also the page for SSA to mention that this is the page to add new medical providers. She suggested adding a required question: new provider, yes or no, and if yes, they need to add, if no, they can move on. Then the facilitator asked her to try adding a new one. The advocate said she would add in “first day of treatment and day last seen” to ensure that SSA requests records back that far. The facilitator asked, “what if we tell you that the DDS will only ask for the past 12 months regardless?” The advocate said it is fine as is, then. Then they returned to the Medical Provider main screen and the facilitator asked what the advocate thinks of the badges (review, needs review, and new). The advocate said she likes those and likes that they’re color coded to show that it needs input or is done. She asked if the “New” was added by SSA or by the claimant? The facilitator said, no, that’s what you just added. The advocate asked, “if the claimant then clicks on New, if it then changes to Reviewed?” the facilitator asked, “what would you expect to have happen?” The advocate said maybe it should then say “new and reviewed” so the claimant doesn’t think they need to do more here.

They moved to Tests screen. The facilitator asked the advocate to look at this screen and note that we don’t have any of those questions about Tests anymore. Then asked her to look at the new screen by clicking “yes” to add one test. The advocate said the “add test” button was obvious enough. She liked the drop-down box for test type. And she liked the drop-down box that give medical providers already in the system, and felt it was all pretty self-explanatory. After they added a new one, the advocate felt that maybe the claimant should need to review the New addition to ensure it is accurate? But the facilitator said we don’t do that, and the advocate accepted it.

Then they moved to the Medicines screens. The advocate liked that there is a list, and the respondent can review the items. The facilitator asked her to try to add a new medicine, which the advocate did. The facilitator asked if the advocate has any comments on the “reason for medicine” text box. The advocate said that it would be helpful to ask them to give the “reason for and condition it is treating.” She also said that side-effects are missing here, and that is something that we should add. The Facilitator asked how would that work, if it’s over the counter? The advocate said she likes the other options in the drop down that say, “no one prescribed” or “don’t know.” The advocate said that the reason for the meds is important to know. She said that some claimants may use other names for meds (brand names or full generic name), and they may take it for a reason other than that medication is normally prescribed. For instance, a muscle relaxer prescribed for migraines instead of pain. She reiterated that we should change to “reason for med and condition it is treating” to the text box.

Then they moved to Other Medical Info page. The advocate liked that this page also includes prisons as a source of information (saying that a bit facetiously). The facilitator asked the advocate to click “yes” and give feedback. The advocate looked and said that it looks fine, because SSA would use a HIPAA release for these anyway. Then clicked next.

That moved them to the Education and Training page. The advocate clicked “yes” and looked at the info needed. She said that, since this is related to info, there should be something in there about how many hours they are attending classes and whether they are online or in person, as SSA will see that as an ability to function (if they have a full course load, they will be deemed no longer disabled). Otherwise, she said it looks good. The advocate then clicked on yes for training and looked at the info needed for that. She said that one looks fine to her. The facilitator also asked her to look at the date for the last disability notice, and the advocate said this is good to know that SSA only needs new info since the last date of CDR.

Then they moved to Support Services screen. The advocate asked if this was duplicative of the prior screen – specifically of the training (because of the IEP). She said the wording needs to be changed here – maybe use “support services/vocational rehab?” The facilitator said it sounds like you’re saying this is redundant – would it be beneficial to put this screen with the training screen? The advocate said, yes, otherwise someone might skip this screen. She also said that the IEP or IPE are all vocational things, and suggested SSA separate out the IEP, even though it is a support service, the bolded portion on this page makes it sound like the respondent already answered this on the page before. She also suggested SSA take out the training from the prior page and incorporate it into this page. The facilitator asked about the IEP – whether it should be here, or with education? The advocate said she is less concerned about that one, but feels that this page is related to training, and those should be together. The advocate chose to go back and see how the questions are phrased on the page before – she said having “vocational training” on the page before will make someone think they already answered it. She suggested SSA take out “vocational training” on the Education page, so that when someone goes to the next page it’s not asking about vocational stuff again. The advocate then clicked “yes” and “add” to see the next page. The advocate liked the check boxes and liked the page.

They then moved to Daily Activities screen. The facilitator asked the advocate to be candid on this page and said that the changes are minor here. The advocate said maybe rephrase to “provide details on how you spend your typical day (not your worst day and not your best day).” She said that this is important information to have – the time they get up in the morning and the time they go to bed at night. She also looked at the check list – the advocate said that we need to add in “reaching overhead” or “reaching above shoulder height.” If you have a hard time reaching, it depletes the job market – many jobs require someone to reach overhead. She then went back to the hobbies and interests – and she said that there needs to be a follow up here on whether they still do these activities as a secondary question. We need to ask, “which of the above do you still participate in?” That can show that they used to have an active lifestyle that they can’t do anymore. She said this is very important to show if they are not still doing it (or need help doing it). The facilitator asked for any other comments. The advocate said to add one more to the checklist: being able to complete a task/stay on task – she said that this needs to be phrased so it is not confusing for the respondents. Maybe call it “completing tasks in a timely manner.” The facilitator said that it was removed as it is not really a task. The advocate said, that just because someone cooks, doesn’t mean that they do it the way others do – maybe they have trouble making it, so they spend 5 min at a time prepping/cooking, so it might take hours to cook, and not do it in a normal timeframe. SSA needs a glimpse of that. Also, some of these disabled respondents have made accommodations for themselves for so long, that they might not remember to mention it. Maybe add in a separate question: do you complete tasks in the same way now as before your disability (takes longer, break it up into steps, etc.), and if they say no, give a drop down box that gives options (takes longer, break it up into steps, etc.). The facilitator asked, “what if they were always disabled?” The advocate said maybe reword it to “do you complete tasks the same way as others in your household/community?” The facilitator also pointed out that we don’t ask how these abilities have changed since the last time they contacted SSA. The advocate agreed, and said that there may be a change, and that needs to be recorded here by asking specifically if they do things the same as a “normal” person without saying “normal,” as we don’t like to use that term for disability.

Next, they moved to Work screen. The facilitator aske the advocate to select “yes” and review the screen. The advocate noted that it did not make the respondent answer further questions. She said we need that info to ensure they can stay on the rolls. The advocate questioned why they ask but do not obtain answers? The facilitator said they are asking here to send a form later to collect that info. The advocate was surprised that SSA is not asking it – not capturing the name of the employer, at the least, to match things up for consistency’s sake. The facilitator asked, “should they capture the name of the employer if someone is still working or if they stopped?” The advocate said “yes” to both.

Then they moved to the Remarks screen. The advocate said it looks fine. She asked what happens if we click next? And expected a summary and opportunity to edit.

They moved to summary screen. The advocate liked that each section can be edited separately. She suggested that there needs to be clear language here that until they hit the Submit button, the form is not actually submitted, and that info should be at the top of the page, or a window that appears to the side regardless of where they are in the summary. She said that people will skip the box at the top of the page, which is why that needs to be down at the bottom as well with further explanation of what the “Save and exit” will do also – that submit is the important one, and maybe Save and Exit needs to also say parenthetically that this button does not send form to SSA – or a pop up that says that to make it clear that the form isn’t sent if they click that button.

Then they moved to Submit button. The advocate liked the pop up regarding what happens when they submit. The advocate liked the options to save and print the PDF of the CDR form and the electronically signed SSA-827. She also reviewed the SSA‑827 to see that it was eSigned. The facilitator asked if the advocate thinks that someone will save or print the form. The Advocate said that most people will not save or print it. She also said that it is next to impossible trying to obtain a form that was submitted to SSA, so it’s best to have copies, but SSA doesn’t give access until there is a hearing and then the rep payee gets a copy of the file. This is why she said at the beginning that the rep payee needs to be informed as well, if this is something that the claimant must fill out on their own. The facilitator asked if this would be useful for someone to be able to access it again, once it’s submitted. The advocate said, yes, it should be saved as a PDF file that can’t be changed, but it should be accessible to print.

They clicked Done, and the advocate said that she liked that the page then shows the CDR as done.

The facilitator said she had a couple question now that we finished:

Thanked the advocate for the feedback, and asked for overall thoughts:

The advocate gave kudos to SSA and said this is much better than anything from before. And approved that SSA has gone to the reps and the public for comment and has taken suggestions. She said there is more to change, but there needs to be some direction to the client. She noted that all forms once submitted are final, if you feel the need for legal counsel, now is the time to obtain it (before you start/submit). She feels that people really need help with these forms and should get that at the outset.
She said maybe when they click on the initial CDR, then there should be an initial question: “do you have an attorney” and if they say “yes” then it should direct them to contact their attorney, and if they say “no” then it should go forward. SSA should be talking to attorneys if the claimant has one, and SSA should already have that info on record.
- Or, at least, “if you have concerns, contact your attorney before completing this form” might be a more viable option

The Facilitator aske the advocate for any other feedback on the application:

The advocate had a question: once someone submits this info, how quickly will it become resident in their case so that it becomes accessible to their rep payee on ERE? She wants access to these files as the attorney.
- Facilitator did not know the answer to that question
She also feels that these should be available on mySSA for the claimants to review as a static PDF (unchangeable).

The facilitator asked: How should SSA notify about a CDR?

The advocate said, are you asking if the respondent will have one, or to tell them that it is available online?
- She said don’t send out more paper – send it as an email or send it through mySSA to flag it. And don’t send it out to everyone, just to the one who needs to be flagged.
- Maybe send out a paper notice only to those people who are up for a CDR
- Or, do a PSA regarding the new online iCDR
- Since you can’t rely that someone who has a mySSA account actually has computer access, definitely send out a notice by mail, but make it clear on the envelope that there is info in the letter that they need to act on.

The facilitator asked: how long it takes to complete a CDR on average:

The advocate asked about the form itself. She said that the forms have too much detail, and it takes at least two hours for the form itself. To gather the info, and until SSA acts on it, it could take up to 2 years (this includes after they have submitted and until the claim has been completed). She said, average time is 4-6 months.

The facilitator thanked the advocate for her feedback and opened the floor to observers.

On observer asked if the 2 years the advocate mentioned is only for gathering info and completing the form – and the advocate said, no – that’s mostly the time for SSA to process the CDR, it takes on average about 2 weeks to gather the info and complete the form.

The advocate thanked SSA for this opportunity to give feedback.

NOTE: The facilitator did not do the survey for this one, as it went late (by 10 minutes). We had no observers from OMB.

Session from 4/1/22:

9:00am Session

Participant: Alan P., from Disability Advocates

The facilitator explained that these screens are on a new platform, and that, while it looks fully formatted, we are still looking at a prototype which SSA may still need to revise. The facilitator then explained the process to the advocate and let him know that we are looking for feedback, especially candid feedback so we can improve the screens, and that we will record the session and take notes but will never include her name in our reports. Then asked if they have verbal consent to record the session, and if the advocate has any questions before starting.

The facilitator then said that we are working with a basic scenario here with John Smith receiving a letter that the CDR is due and can be completed online. Then the facilitator started to go through the screens with the advocate. The advocate stated that accessing mySSA is getting easier to use by his clients, but still can be made easier. But he reiterated, mySSA has definitely gotten more user friendly and is more accessible over the past year. Alan had no questions or concerns on the main screen page.

Next, they looked at the access screen. The advocate had no questions, and understood the right to decline, and understood it will then be completed by paper and not electronically. He thought the language and highlighted sections were fine. Alan’s one comment on the agree or disagree to electronically sign page was to write out the form names next to the form numbers. He stated he didn’t know form numbers off the top of his head but knows the names of the forms.

They then moved onto the Information about You page. The advocate liked the revisions to the questions. They make it clear that these questions are related to medical and school records. He thought the questions looked fine. He wasn’t sure if the question about can you write a simple question would capture people with a learning disability, but that would also already be a part of the medical records. Therefore, he thought the questions were fine.

On the Someone we can Contact page, the advocate recommended changes to the relationship to you drop down. He thought there should be a spot to specify relationship of family member option.

Next, they moved onto the Medical Conditions page. The advocate had no questions or comments and moved through the page without issue.

On the Medical Providers Page, the advocate liked that the chart is prefilled. He really liked that. He understood that we were asking only for someone seen in the last 12 months and thought that was fine as well. The facilitator explained that a lot of questions were removed from the provider page, and the advocate agreed that what’s shown is all that was needed. The facilitator agreed and stated that DDS said that the information now on the screen is really all that was needed. Again, the advocate agreed and liked the new page. His one recommendation is to add a field for as best as you can recall seeing the medical provider, instead of just can’t remember. After updating, he really appreciates the flags and the chart. He recommended maybe adding another field to the chart of last contact/report date, to maybe put a time stamp on each provider, to help the claimant remember. The advocate was hesitant to delete providers in the chart that they haven’t seen in the last 12-months, just to have them on record, but doesn’t know if claimants would delete due to the instructions above. He recommended possibly adding a field where a claimant can add dates last seen/visit, instead of deleting inactive providers. On the adding a new provider screen, the advocate recommended adding a field as to why you saw this new provider (second opinion, new issue, connection of a lawsuit, etc.), and what kind of Dr/Provider they are.

On the Tests page, the advocate asked why we had blood test (not HIV). The facilitator told him he’s not the first person to say that but couldn’t offer an answer. Beyond that, he didn’t have any additional issues or questions.

They moved onto the Medicines page. The advocate recommended the ability to upload an attachment with a list of all of the medications someone was taking. He thought that would be very helpful and direct. If SSA couldn’t do that, possibly add a place to include pharmacies. That way, if there was an electronic means to request documentation, it would be fast and easy. He stated that a lot of his clients can’t pronounce their medications or spell them. Some instances it’s a guestimate and may not be exact names. On the add new medication page, the advocate liked the setup of the page. He recommended possibly adding a place for side effects, but added, if someone is taking multiple medications would be hard to determine which is the culprit, but still thinks having a place for side effects would be helpful.

On the Other Medical Information page, advocate said this page is straight forward and sufficient. He had no additional comments.

Next, they reviewed the Education and Training page. Advocate stated yes, he would like to capture these questions, but 99 times out of 100, the answers are just no. He had no additional questions or comments.

On the Support Services page, the advocate didn’t have any questions or comments on the page. On the add plan or program page, the facilitator stated that the types of services question at the bottom used to be a text field, but now it’s check boxes. She asked if he had any comments. The advocate recommended adding an option for medical evaluation/examinations. Besides that, he understood why the questions are asked, and agreed with them.

Next, they moved to the Daily Activities page. The advocate stated that when an applicant files, they typically get and fill out activity reports and other forms of that nature. He thinks that being able to input whatever in the describe your typical day field would yield all sorts of things. If an applicant fills it out, you could get one thing, if an attorney fills it out, could be a completely different excerpt. He doesn’t think that the describe your typical day field is really needed. The function reports should be all that is needed in regard to this information. He likes the bottom options to explain difficulties that were checked. Thinks that field is necessary. Other than that, didn’t have any additional comments on the daily activities page.

On the Work Page, the advocate stated that they should request when did you work, how long did you work, name of employer, and why did you stop. That should trigger additional forms needing to be filled out. Or someone in SSA should be able to pull Wage Reports and verify the information. The advocate just thinks being able to input more information here would speed up other processes.

They then moved onto the remarks screen. The advocate had no comments or concerns with the remarks.

The advocate liked the Summary Page. He liked the opportunity to review and edit anything where needed. The advocate asked if he submits, can he print it out. He would like the option to do so. After submitting, he liked the next screens, and the ability to print the completed electronic signed form. After we submitted and completed the CDR, the facilitator asked if the advocate could see how the main page was updated. The advocate didn’t see the revised language. He recommended adding bold or highlight that the CDR was accepted. He again stated, regardless of if it was done when hitting submit or not, they should be able to print out the form.

In conclusion, the advocate said he would like the following:

The ability to upload/attach evidence or additional forms that would be helpful for the review;
- If there was a report stating applicant isn’t better from the medical provider, or a list of medications still being taken.
  - That may help to get the information right into the system when starting the CDR
He would like signing up for mySSA to be easier and more user friendly;
He thinks a lot of his previous comments in round 1 have been considered and changed. He appreciates that;
He wishes he can see the applicant’s entire file from the beginning, when it’s pending with a state agency. Until the process goes to the hearings level, he is unable to see the entire case file;
- He doesn’t understand why things are hidden at different stages
  - He should be able to see determinations
He would like to be able to CDR Appeals online.

The facilitator asked the advocate how long it typically takes them to complete a CDR. The advocate stated that completion of the form would typically only take an average of a 1/2 hour. As far as how much time it would take to gather the information and such, he stated that the clients tend to do that prior to reaching out to him, so he wasn’t sure.

The facilitator thanked the advocate and ended the session.

NOTE: The facilitator did not do the survey for this one. We had no observers from OMB

Usability Testing with SSI and SSDI Recipients (4/4/22 – 4/8/22):

Sessions from 4/4/22:

9:00am Session:

The facilitator then asked the beneficiary to tell her where he would start to get to the CDR. He said he would go to Action Required, as that seems the most urgent. The beneficiary had no problem clicking on the correct link to get to the iCDR.

The facilitator then asked the beneficiary to look at the Medical Release page. The beneficiary felt it was pretty straight forward. The facilitator asked the beneficiary what he thinks would happen if he clicked on decline, rather than agree, and the beneficiary felt that it would likely not allow the respondent to continue online. The facilitator asked him to click on decline and read the text box. The beneficiary felt that the text box was a bit confusing, as he did not feel it was clear that someone would need to do both the medical release and the CDR on paper if they decline on this page. The beneficiary also said that he thinks that when someone mails in their forms, that SSA saves everything in the file, and feels that it shouldn’t be an all or nothing here, there should be a way to allow someone to do the CDR form online, even if they choose not to sign the medical release form online. The facilitator then asked him to change the response to “agree” and look at the blue text box. The beneficiary had one comment: he felt this screen should tell people how it will save the info and how someone might be able to save the form later. The facilitator asked him to hit next to move on.

They moved to the Instructions (gathering info) page – and the facilitator asked the beneficiary to review it and let her know if he has any comments. The beneficiary felt that it was pretty cut and dry but felt there should be a chat feature in case someone needs help while filing out the form. Otherwise, he felt it was fine.

Then they moved to the Information About You page, and the facilitator reminded the beneficiary that we are using the John Smith scenario here and said that he should go through and fill in the form on this page here, and that he is welcome to do so on his own or ask for responses from the scenario. The beneficiary filled it out, and then the facilitator asked him to comment on the page. He felt it was pretty easy to follow and complete and did not feel that anyone would have issues with it. The beneficiary also asked if he should be saving as he goes along and suggested that there should be another option on the page for “save and continue” instead of just “Save and exit” and to let people know on each page that they can save and continue, or if they want to exit and come back later, they can save and exit.

They moved to the Someone We Can Contact page. The facilitator gave the scenario info that John Smith has a brother and asked the beneficiary to fill out the info. She warned the beneficiary here that the tab button does not work well in the prototype. The beneficiary felt it was pretty easy to understand and fill out. The facilitator asked, “what do you think that SSA is asking for with the contact person?” the beneficiary thought that would be someone who might have more info, or be someone who helps set up appointments or things like that.

Then they moved to the Medical Conditions page. The facilitator gave the scenario info to the beneficiary but said he could make up his own answers if he chooses instead. The beneficiary completed the page, then the facilitator asked for feedback. The beneficiary thought it was pretty easy and made sense.

They moved to the Medical Providers page. The facilitator asked the beneficiary to read through it after choosing “yes.” Then she asked what do you think is going on here? The beneficiary said he assumes this is the info from the last review, or the last time he gave info. He asked if he could click on “review,” and the facilitator said “yes.” The beneficiary reviewed the next page on the first medical provider and asked if there is an edit button. The facilitator said, no, you should be able to just click on the fields to fix them, if needed, and asked what he thinks of it. He said that that looks good, then. He said that he was glad that it removed most of the dates, because those dates are hard to remember. He also feels that those dates should also be transferred to this if his doctor has sent the medical info to SSA. He said it is hard to remember dates or remember where the paperwork is for medical appointments. He also felt that this page should also have a “save and continue” button.

They returned to the Medical Providers page, and the facilitator asked if he thought he successfully reviewed the medical info for the first entry, and he said “yes,” because it says “Reviewed.” However, he was unsure if it was complete, as he might still need the dates for the medical appointment, and then it would be reviewed, but not complete, even though he had chosen the “I don’t remember” option. He thinks most people would click on “I don’t remember” because it’s easier than trying to look anything up. The facilitator asked him to look at the full list again and asked what he would do if he hasn’t visited the last entry in over two years. The beneficiary felt that there should be a check box asking if anything has changed over the past 12 months, and, if the beneficiary clicks “no,” they shouldn’t need to check anything, and if they said “yes,” then they should need to review. And he felt that dates shouldn’t matter or be relevant and shouldn’t need to be part of the record here. The facilitator then asked him to delete the last provider to see if he can navigate it. The beneficiary asked, “once I delete it, does it stay on the file and say “deleted,” or does it drop off the file?” The facilitator asked what he thinks it should do. The beneficiary said that it should ask “is this still one of your doctors? And you click “yes” it stays, and if you click “no,” it should stay but become inactive so that it stays in the file.” The facilitator said that this hospital was provided during the last review, and SSA only wants the information from the last 12 months, so if you delete it here, it would be gone. The beneficiary felt that it should go inactive, and then if it becomes active again, allow for reactivation. The facilitator then said, in the scenario, we have a new medical provider, how would you include that? The beneficiary clicked on “add medical provider” and included the info the facilitator gave him. He asked if he needed to include the full address, and the facilitator said, no, but asked him what would help? And he said it would be helpful if there was a drop-down box if the doctor is already in SSA’s system. It should allow someone to click on her and allow for dropdowns to fill in the rest. He had no further comments.

Next, they moved to Tests screen. The facilitator said we will say that Dr. Hammond ordered an x-ray. The beneficiary said “yes,” but felt that there should be a drop‑down box here to give a list of tests, as someone might not think about what a test might be here. He said it would give them more info on what they need to add. For instance, a medical examination might be considered a test, so it would be easier for someone to see examples to see what SSA considers a test, then someone could just click on it to add it. The facilitator asked him to look at the drop down after clicking on “add test.” The beneficiary felt that was a good drop-down box, but still thought it should be earlier. He felt that the screens were easy to fill out, though. The facilitator asked about the “NEW” label on the screen. The beneficiary said he assumed it was because he just added it, and said it makes sense.

They moved to the Medicines screens next. The facilitator said that it looks similar to the medical providers page and asked if it makes sense. He said yes. She asked him to add in a new medicine, Vitamin B3. The beneficiary clicked on the screens and added in the info and felt that it was nice to have the drop-down box with the medical provider there. He had no comments on the page and felt it was easy to fill in.

Then they moved to Other Medical Info page – the beneficiary read through it, and he questioned who that person might be, so the facilitator asked him to look at the list below – he felt that the examples should be in a different color to make it pop out better, even if just the word “examples” is in red to make it clearer. The facilitator asked him to click “no” and move on.

They moved to the Education and Training page. The facilitator asked him to read through and give comments. The beneficiary said that it seems pointless and not relevant to the CDR – does education affect the decision? He sees more relevance in specialized job training but feels that neither is really relevant. The facilitator asked if the beneficiary noticed the date in the question, and asked if it was clear what that date is? The beneficiary said “no,” unless that was when I applied. The facilitator asked if it would be better for us to say “since your last review” instead of the date? And the beneficiary said “yes” and suggested it should be consistent across the board.

Next, they moved to Support Services screen. The facilitator asked if this page is clear and if the beneficiary is familiar with TTW and PASS. He said he is familiar with TTW but not with PASS. The facilitator asked him to click on “yes” to move to the next screen to fill in the info about the program, then had him skip down to the part where it asks if the respondent is still participating in that program. The beneficiary filled in the info as given to him by the facilitator, she then asked him to look at the check box list at the bottom and let her know if those look familiar – he said, no, they don’t look familiar. She thanked him and said they could move on.

They then moved to Daily Activities screen. The beneficiary laughed when he saw the page. The facilitator asked him why he was laughing. He said that this still falls into the relevancy thing – I know that SSA asks about hobbies and interests, but he doesn’t really understand the relevance of this question. He said that people still need to function whether they have pain or not, because life requires people to move and deal with responsibilities. He looked at the check boxes, and said he was trying to see obstacles for doing each possible task. The facilitator asked him to tell her what he might place in the last text box about difficulties in completing tasks – he said original diagnosis can affect things in a bad way. The facilitator asked him to scroll back up to the top of the page again, and asked why he thinks that SSA might ask these question – he said that he thinks they want to see what people do, but it doesn’t give any qualifying language to explain if you do them because you have to, or if you have difficulty with these tasks, which is why he sees no need for this information.

They moved to Work screen. The facilitator asked him to say yes here to review, and that John has worked for a few months at the bank but stopped working because of the health issue. Then asked if he has any comments on this page. He said no, not really, and felt it was straight-forward. The facilitator asked him what kinds of things he would put into the text box for additional info – he said only if he has a new condition, but felt that should go in the form earlier, but also liked that it was here.

The beneficiary then moved to the Remarks screen and skipped through it.

The he moved to summary screen. The beneficiary said he liked the screen, because it’s easy to understand and he likes the categories and sub-categories. He said it is his favorite screen. He said if he had to comment it would be that, if dates were needed, it should be something that SSA highlights it in the categories so someone might know if they need to add it in later. The facilitator said that it is okay to say, “I don’t remember” and the beneficiary felt that was fine, then.

They Moved to Submit button. The beneficiary clicked through to the green text box regarding printing. He said that it should automatically save it in the mySSA application and should say that it saved there and will allow you to print it later, if someone wants to see it. He said that he has a printer available, but most don’t, so saving is more important, and he really feels that it should save automatically in mySSA. The facilitator asked him about the two linked docs in blue, and asked which one should be automatically saved? The beneficiary said anything you do here should be automatically saved already, and not need to be printed or saved elsewhere.

They clicked Done, and the facilitator asked him to review the main mySSA page. He said that it was clear that the CDR was done. The facilitator then said that he had said that everything should be saved here on the mySSA page, and asked if he expected to be able to review it later? He said yes.

The facilitator thanked him for his feedback and went into final questions:

Asked for overall thoughts:
- The beneficiary said he liked it. His biggest suggestion is that it is not good that they normally need to put in info that should be there, so he feels that once he put info in at the doctor’s, it should submit to SSA with all the info, and it should not be something the beneficiary needs to add. He felt it should be the same with employers, and some information should be available on mySSA.
- She asked if he has completed it on paper and whether he has ever called SSA for help. He said, yes, he completes it on paper, but SSA only called him for more info as needed.
- She asked how long it takes to complete the paper form – he said he d0oesnn’t remember, but he did need to call for how to fil it out, and SSA called him for info that was missing. He remembered it taking a long time, and it was time-consuming, and he had to pick it up from the office, which was also time-consuming. He said that the paper form is difficult.
- She asked if he would prefer the online version to the paper version? He said yes, because he wouldn’t have to mail or fax it, and he found it was easier to type it in, rather than filling out paperwork for review.
- She asked how SSA should inform him about the next review – he said email him ahead of time, but said SSA should also mail a reminder, and maybe also call and leave an automated method by a specific date (maybe once a month out, and one a week prior to the due date). He feels that more ways to inform them is better.

The facilitator then asked the beneficiary to complete a satisfaction survey.
- The beneficiary responded to the survey

Thanked the beneficiary for his feedback and opened the floor to observers.

One observer asked about the review page, and wanted the beneficiary to clarify that if there was info that was from a previous page that needs to be completed, did you want us to highlight that it needs to be completed?
- He said yes
Another observer asked a quick question: how would you fill out this form? Mobile or desktop?
- He said desktop because of his eyesight, but would love a mobile application option.

The facilitator thanked the beneficiary again and ended the session.

NOTE: We had no observers from OMB.

12:00pm Session:

The meeting started at 12:00 pm, and the facilitator waited until 12:10 pm for the participant to show, before contacting the contractor to see if they knew where the participant was. The contractor stated they talked to the participant earlier to confirm the appointment, but they do not know what happened. The contractor stated they would try to reschedule the beneficiary for another session. We gave the participant a little more time to join the call, but the participant was a no show. We ended the call at 12:17 p.m.

2:30pm Session:

The meeting started at 2:30pm, and the facilitator waited for 20 minutes, but the beneficiary did not attend. Per the contractor who scheduled the appointment, the beneficiary had an emergency that kept him from today’s session. They will reschedule him for a later date this week. We ended the call at 3pm.

Sessions from 4/5/22:

9:00am Session:

The facilitator then asked the beneficiary to look at the Medical Release page. The beneficiary felt it was pretty straight forward. The facilitator asked the beneficiary what she thinks we are asking on this page. The beneficiary stated we are asking for his permission for providers to release information. The facilitator asked the beneficiary what he thinks would happen if she clicked on decline, rather than agree, and the beneficiary felt that it would likely not allow the respondent to continue online, and he might receive something in the mail complete from SSA. The facilitator asked him to click on decline and read the text box. The beneficiary felt that information was clear and understood what was being asked. The facilitator then asked him to change the response to “agree” and look at the blue text box. The beneficiary had no comments. The facilitator asked him to hit next to move on.

They moved to the Instructions (gathering info) page, and the facilitator asked the beneficiary to review it and let her know if he has any comments. The beneficiary felt it was clear, and stated we are asking this information to determine if the person still has a disability.

Then they moved to the Information About You page, and the facilitator reminded the beneficiary that we are using the John Smith scenario here and said that he should go through and fill in the form on this page here, and that he is welcome to do so on his own or ask for responses from the scenario. The beneficiary filled it out, and then the facilitator asked him to comment on the page. The facilitator asked the beneficiary what he thought about the question “Can you read a simple message in the language you selected above?” The beneficiary said that if he selected “no” then they would not be able to complete the form. The facilitator asked him to change to another language. The facilitator asked what he thinks will happens when she selects another language, and the beneficiary stated it will change the form to the new language, or maybe someone from SSA who speaks that language will call, or SSA may send a form in the form in another language.

They moved to the Someone We Can Contact page. The facilitator asked, “what do you think that SSA is asking for with the contact person?” The beneficiary thought that would be for case workers, case mangers; therapists or social workers, The facilitator stated SSA might need to provide more explanation. They moved down to the “Relationship to you” dropdown option, and the beneficiary said the list is very detailed. The facilitator asked the beneficiary to select brother from the dropdown option.

They moved to the Medical Conditions page. The facilitator gave the scenario information to the beneficiary and asked what he thought about the question “what assistive devises are you currently using if any?” The beneficiary thought it was very detailed, and said it is good that we included the other option. The facilitator asked what he thought would happen if he selected the “other” button. The beneficiary replied that it would ask him to provided other information. The beneficiary completed the page, then the facilitator asked for feedback. The beneficiary thought it was pretty easy and made sense.

Then they moved to the Medical Providers page. The facilitator asked the beneficiary to read through it after choosing “yes.” Then she asked, “what do you think is going on here?” The beneficiary said it makes sense, and we may want to know the name, address of the providers, or why the claimant went to the provider. The facilitator asked the beneficiary what he thought we were trying to ask on the “Review and update Medical providers” section. The beneficiary stated it shows the medical providers they have gone to, and the “need review” portion indicates that he needs to update the information . The facilitator asked if he thought it made sense to show providers from the last time, the beneficiary responded yes. The beneficiary clicked “review,” and stated that he needed to update if the information changed. The facilitator asked him to put in a date, and asked if he remembered the last time he saw his provider. The respondent replied that he knows when he sees his provider, because he sees them 2 to 4 times a week. He said that this is what he was expecting when he saw the questions. The facilitator asked if he could tell that the information was reviewed, the beneficiary stated he could tell it was reviewed, because it went from red to blue. The beneficiary also stated that it is good that it changes colors, because people know that red means stop, and green means go. The facilitator asked the beneficiary what he would do if someone no longer had the listed provider shown on the page. The beneficiary replied he would click the “delete” button. The facilitator asked the beneficiary what he thinks he should do for the new doctor button. The beneficiary stated he would add a medical provider, then the beneficiary added a new doctor using the button. The facilitator asked him if he noticed that the new doctor was added, the beneficiary said yes, it is now in blue.

Next, they moved to Tests screen. The facilitator asked the beneficiary what the medical test page tell him, and the beneficiary stated that it could be asking for blood work, x-rays MRIs, etc. The beneficiary clicked on yes, and then “add test,” and the facilitator asked him if this what she was expecting. The beneficiary said yes, it depends on what condition the person has to determine what tests are ordered. The beneficiary also stated it is very good that we have the “other” option to add additional conditions. The beneficiary said this page is clear to him.

Then they moved to the Medicines screens. The facilitator asked if it looks similar to the medical providers page, and asked if it makes sense, and the beneficiary said yes. The facilitator asked the beneficiary what he thinks we are trying to ask. The beneficiary said he thought we would ask for medications, the milligrams of the medicine, and how often they take the medicine. The facilitator asked the beneficiary what he thought about having medications to review, the beneficiary stated that he likes that we have that option. The facilitator asked him to add in a new medicine, Vitamin B3. The beneficiary clicked on the screen, and said he is usually asked to add medications, but it usually has a chart listing the medications, the milligrams of the medicine, and how often he takes them. The beneficiary said it was not what he was expecting , because he thought there might be a chart, but stated that this screen may be easier for some people, because they may not know the milligrams of the medicine, and that might be good for older people. The facilitator asked him what he thought about the reason for medicine section, and the beneficiary stated he understood what it means, and would not change anything on this page.

They moved to Other Medical Info page. The facilitator asked the beneficiary if he read through it and asked if the question made sense. The beneficiary stated that it makes sense, and it is good that it has examples listed. The facilitator asked him to click “no” and move on.

Then they moved to the Education and Training page. The facilitator asked him to read through and give comments. The beneficiary suggested SSA change the language to “Since your last disability review” instead of “Since your last disability decision.” He had no other comments.

They moved to Support Services screen. The facilitator asked if this page is clear and if the beneficiary is familiar with TTW. He said he is familiar with TTW. The facilitator asked him to click on the ticket to work link which took him to the SSA.gov page detailing TTW. The beneficiary said it is a good idea to have the option to look and the program and that the page explains the program. The facilitator asked him to click on “yes” to move to the next screen to fill in the information about the program, then had him skip down to the part where it asks if the respondent is still participating in that program. The beneficiary filled in the information the facilitator provided. The facilitator then asked him to look at the check box list at the bottom and let him know if those look familiar. The beneficiary said it looked kind of familiar but stated the fact that we always have “other,” as an option because people can’t list everything. He thought the page looks good and had no concerns.

Then they moved to Daily Activities screen. The facilitator asked him to tell her what he might place in the last text box about difficulties in completing tasks, and he said the original diagnosis can affect things in a bad way. The facilitator asked him to scroll back up to the top of the page again and asked why he thinks that SSA might ask this question. He said that he thinks they want to see what people do, but it doesn’t give any qualifying language to explain if you do them because you have to, or if you have difficulty with these tasks, which is why he sees no need for this information.

Then they moved to Work screen. The facilitator asked him to say “yes” here to review and told him that John [from the scenario] has worked for a few months at the bank but stopped working because of the health issue. Then the facilitator asked if he has any comments on this page. He said no, not really, and felt it was straight forward. The facilitator asked him what kinds of things he would put into the text box for additional information. The beneficiary said only if he has a new condition, but felt that should go in the form earlier, but also liked that it was here.

They moved to the Remarks screen and skipped through it.

Then they moved to the Summary screen. The beneficiary said he liked the screen, because it’s easy to understand and he likes the categories and sub-categories. He said it is his favorite screen. He said if he had to comment it would be that, if dates were needed, it should be something that SSA highlights it in the categories so someone might know if they need to add it in later. The facilitator said that it is okay to say, “I don’t remember” and the beneficiary felt that was fine, then.

They moved to Submit button. The beneficiary clicked through to the green text box regarding printing. He said that it should automatically save it in the mySSA application and should say that it saved there and will allow you to print it later, if someone wants to see it. He said that he has a printer available, but most don’t, so saving is more important, and he really feels that it should save automatically in mySSA. The facilitator asked him about the two linked docs in blue, and asked which one should be automatically saved? The beneficiary said anything you do here should be automatically saved already, and not need to be printed or saved elsewhere.

The facilitator thanked him for his feedback and went into final questions:

The facilitator asked for overall thoughts:
- The beneficiary said he liked the screens. His biggest suggestion is that it is not good that they normally need to put in information that should be there already, so he feels that once he put information in at the doctor’s, it should submit to SSA with all the information, and it should not be something the beneficiary needs to add. He felt it should be the same with employers, and some Information should be available on mySSA.
The facilitator asked if he has completed the CDR on paper and whether he has ever called SSA for help.
- He said, yes, he completed it on paper, but SSA only called him for more info as needed.
The facilitator asked how long it takes to complete the paper form?
- He said he doesn’t remember, but he did need to call for how to fill it out, and SSA called him for information that was missing. He remembered it taking a long time, and it was time-consuming, and he had to pick it up from the office, which was also time-consuming. He said that the paper form is difficult.
The facilitator asked if he would prefer the online version to the paper version?
- He said yes, because he wouldn’t have to mail or fax it, and he found it was easier to type it in, rather than filling out paperwork for review.
The facilitator asked how SSA should inform him about the next review
- He said email him ahead of time, but said SSA should also mail a reminder, and maybe also call and leave an automated method by a specific date (maybe once a month out, and one a week prior to the due date). He feels that more ways to inform them is better.

The facilitator then asked the beneficiary to complete a satisfaction survey.

The beneficiary responded to the survey

The facilitator thanked the beneficiary for his feedback and opened the floor to

observers.

On observer asked about the review page, and wanted him to clarify that if there was info that was from a previous page that needs to be completed, did you want us to highlight that it needs to be completed?
- He said yes

Another observer asked a quick question: how would you fill out this form? Mobile or desktop?
- He said desktop because of his eyesight but would love a mobile app option.

The facilitator thanked the beneficiary again and ended the session.

NOTE: We had no observers from OMB.

12:00pm Session:

The facilitator then asked the beneficiary what the beneficiary would do to start the CDR. The beneficiary stated that she would hit the action item to start the CDR.

First, they went over the Medical Release Form for the CDR Report. The beneficiary stated she understood the need for medical forms, as that would explain why someone would still be disabled, but she didn’t understand the need for education. The facilitator asked the beneficiary to try and decline the electronic form and explain what she thinks. The beneficiary said she doesn’t know why someone wouldn’t want to do this electronically, but it if they don’t, they would have to complete the paper form. The beneficiary had no additional questions on the first page.

Next, they moved onto the Instructions page. The beneficiary thought that there was a lot to read and go over on this page, but after review, she didn’t think it was actually bad. She stated that her doctors would have a majority of the information needed. They provide her with a packet after each visit. The beneficiary didn’t have any additional questions on the page.

On the Information about You page, the beneficiary had no problems answering the questions. The facilitator asked the beneficiary why she thinks we ask the language questions. The beneficiary stated that it only seems right to ask, as English is a second language for many people. She stated that if you select a different primary language, that either a translator is provided, or the rest of the form should change to their preferred language. She wasn’t sure why we asked the last two questions.

Next, they reviewed the Someone We Can Contact screen. The first thing the beneficiary asked was what if the applicant didn’t have anyone. She stated that she does, but some people might not. The facilitator asked if the beneficiary thought the yes or no would suffice if someone didn’t have anyone. The beneficiary said that to help the applicants case, she felt like someone would need to be input in this space. The facilitator asked if the drop-down options are ok. The beneficiary said that the options look good. The beneficiary commented that if they don’t speak English, that the option for an interpreter should also be on this page.

On the Medical Conditions page, the beneficiary had no problems inputting the information. The facilitator asked what the beneficiary thought about the last question and options. The beneficiary had no questions or comments, thought that the options covered the main ones. In addition, the option to input “other” is good.

Next, they reviewed the Medical Providers page. The beneficiary had a hard time understanding where the providers came from. The facilitator had to explain that these are for a fictitious beneficiary. She explained that this screen is where your providers would show up if this was your own record. After that, the beneficiary understood, and liked the set up and options. The beneficiary stated that thinking of the exact month and year of the last visit would be difficult. However, she believes that she can go onto her own portal and pull that information. She just said that it would be difficult off the top of her head, but she could find it. The beneficiary said if she couldn’t find exact information, she would just estimate it. The beneficiary liked that there is an option for “don’t remember” but would rather estimate than give nothing. After submitting the updated provider, she could tell right away that her update processed in the chart. The beneficiary liked that. The facilitator then asked the beneficiary to add a new provider. The beneficiary had no problems adding a new provider, nor had any questions. Once submitted, she saw the new provider right away on chart. There were no additional questions or comments on the page.

Next, they moved onto the Tests page. The beneficiary said that she would like to see more information on top of the page. She stated that maybe they can explain what kind of tests leading into the drop-down selections. The facilitator asked if the options looked good, and the beneficiary stated that it looked good, and had no additional comments.

On the Medicines page, the beneficiary said that this page seems self-explanatory. Again, she stated that she likes the chart and options. The facilitator asked the beneficiary to add a new medicine. On that page, the beneficiary had no problems, and had no questions or concerns.

Next, they moved onto the Other Medical Information page. The facilitator asked the beneficiary if this screen made sense to her. The beneficiary stated that it did make sense. She thinks that this page pertains to other benefits, or some other place you have to describe your disability. The facilitator asked if the beneficiary thought she could answer this confidently by herself, and the beneficiary said she thought she could, but her answer would be no.

On the Education and Training page, the facilitator asked the beneficiary what types of training she thought they were asking for. The beneficiary didn’t know. The facilitator said that it was ok to move forward, and the beneficiary just selected no to both questions.

Next, they reviewed the Support Services page. The beneficiary understood what was being requested and said it all seems self-explanatory. The facilitator asked the beneficiary to add a program to review the next questions. The beneficiary said that she had no comments or concerns with the questions.

On the daily activities page, the beneficiary asked how specific should they be? Some people may say a couple activities, and some may do a play by play for an entire day. The facilitator asked the beneficiary why SSA would want to know this information. The beneficiary stated that these questions are probably trying to gauge what exactly someone claiming disability can do. She thinks that it would be important to explain and provide as much information as possible to help your case. The beneficiary liked the options but said that explaining them at the bottom of the page is a lot to do. She said that most people would only provide detail for a couple of the above instances.

Next, they moved onto the Work page. The facilitator asked what the beneficiary thought about the page. The beneficiary said that it looks self-explanatory, no questions or comments.

On the Remarks page, the beneficiary had no questions, and stated that she would probably just leave it blank.

Last, they reviewed the Summary page. The beneficiary really liked the summary page but had no questions or concerns. The facilitator asked the beneficiary if she was happy with all of the information, what she would do to submit. The beneficiary stated that she would hit the submit button. The beneficiary then clicked Done, and the facilitator asked her to review the main mySSA page. The beneficiary didn’t immediately see the CDR was accepted, but said she knew it was complete. The facilitator pointed the updated language out for her.

The facilitator thanked her for her feedback and went into final questions:

The facilitator asked for overall thoughts:
- The beneficiary said she liked the application, and that it seemed easy for her

The facilitator asked the beneficiary if she would be comfortable completing the CDR online. The beneficiary stated that she would definitely complete the CDR electronically. The facilitator then asked if she felt comfortable doing it on her own. The beneficiary stated that she definitely felt comfortable doing it by herself.

The facilitator asked the beneficiary how she would like SSA to notify her of the need for a review. The beneficiary stated that she would love notification via email.

The facilitator then asked the beneficiary to complete a satisfaction survey, and the beneficiary responded to the survey

The facilitator thanked the beneficiary for her feedback and opened the floor to observers.

One observer asked the beneficiary: on the first page, pertaining to the questions about languages, you mentioned if the person doesn’t speak English, you expect to see questions about an interpreter, or do you expect SSA to contact you?
- The beneficiary stated that she thinks SSA should provide an interpreter for you. In addition, she thinks the form should change into whatever your preferred language is.

The facilitator thanked the beneficiary again and ended the session.

NOTE: We had no observers from OMB.

2:30pm Session:

The facilitator thanked the beneficiary for joining us and informed him that we are working with and online version of the CDR, and working with both beneficiaries and advocates to get feedback. She then said we are working with a basic scenario here with John Smith receiving a letter that the CDR is due and can be completed online. She let the beneficiary know that John would have received a letter which would say he could go to his mySSA account to complete the CDR. She asked if the beneficiary ever uses his mySSA account, and he said he rarely uses it. She then asked the beneficiary how he would start to complete the online CDR. He said he would click on the link where it says it needs action.

Then they looked at the Medical Release page. The beneficiary said he understood what it was and that it looks easy to understand. The facilitator asked him what he thinks would happen if he chooses to decline. He said that declining would not give his permission for medical records. She then asked him to click on decline and read the text box and let her know what he thinks. He said that he understands that he can still submit it, but not online. The facilitator asked him to tell us his options if he declines. He said that it means he is done here and would need to do this offline. She asked him to switch to “I agree” and give his feedback. He said that it looks self‑explanatory.

They moved to the Instructions (Gathering Information) page, and the facilitator asked the beneficiary to look it over, then asked him for comments. He said that it just looks like an information page. She asked if the information makes sense, and he said, yes. And she asked if he had any concerns or thought anything was missing, and he said no.

Then they moved to the Information About You page, and the facilitator asked the advocate to look over and give his feedback. He said he was just looking over the options. She asked him to complete the page and talk about it. He said it all seems self-explanatory. She said we can say that John Smith is fine with English and asked if the beneficiary has any questions so far. He said it looks easy to understand. The facilitator asked, “what if John couldn’t speak English, what do you think would happen?” The beneficiary thought it would switch to the language John chooses. He thought this was easy to understand, also. The facilitator then told him that it will not switch languages and asked for his feedback. She asked him why they might be asking about language. He said maybe so that someone can translate for the respondent.

Then they moved to the Someone We Can Contact page. The beneficiary felt this page was easy to understand. The facilitator asked him to fill it in using the John Smith scenario. He did and said that he had no comments as it was easy to do. The facilitator asked if he had any other comments on this page. He did not.

Next, they moved to the Medical Conditions page. The facilitator gave the scenario about John’s conditions and asked how the beneficiary would complete the page. He said he would list the conditions and did so. The facilitator asked if he would include height and weight, and he said he would. The facilitator asked the beneficiary to look at the list at the bottom and let her know if he has any comments. The beneficiary said no. They added in that John Smith needs a back brace, then moved on.

They moved to the Medical Providers page. The facilitator asked the beneficiary to look at the question and let her know if it makes sense. The beneficiary said it makes sense. The facilitator then gave him info for the scenario, and had him fill it in. Then she asked about the information that popped up on the screen and asked him how he thinks that got in there. The beneficiary said that it looks like it’s from the last time the information was provided to SSA. The facilitator pointed out that this information could be a few years old, and asked if he thought that was right, or if this should be info that was just added in a few days ago. The beneficiary said that he expected it would be older info from the last review. The facilitator asked him what he would do with this info. He said he would want to review it to make sure it was still up to date. She asked him to review the first one and had him review the info and give feedback. He had no comments. She asked him to look at the date of the last appointment and asked if that seemed like something he would be able to provide. He said that he keeps all of his appointments on his calendar and would be able to look it up. The facilitator asked if he had any comments on this review form. The beneficiary said he doesn’t have comments, as it is easy to complete if you have the information. She asked if he thought he needed to add in a date there, and he said no, because there is an “I don’t remember” option.

They then returned to the Medical Providers page, and the facilitator asked if he noticed any changes. The beneficiary said he noted that the top one said it was reviewed, and he liked that it updated like that. The facilitator asked what he would do if any of the medical providers were no longer relevant. The beneficiary said he would hit the delete button. She asked him to do that, and he said it was easy enough to do. She then asked him how he might add a new doctor, if he had one. He said he would hit the button that says, “Add new medical provider.” She asked him to click on it and then asked if it was easy to understand, and he said, yes, it looks the same as the other screen. She asked him to hit save and move back to the other screen. She asked if he saw the new entry, and he said, yes, with the status that says “new.”

Then they moved to Tests screen. The facilitator asked what he thinks this screen is about. He said it’s asking about medical tests in the past 12 months. She asked if it seemed clear to him, and he said, yes. She asked him to try adding in an x-ray for John Smith. He hit “add test” and the facilitator asked him to review the drop-down box and give feedback. He said that the list looks self-explanatory. The facilitator asked him to fill in the rest of the screen and then asked if it made sense. He said yes, it does.

Next, they moved to the Medicines screens. The facilitator gave the beneficiary a moment to read through and asked for feedback. He had no comments. She said we can say “yes” that John Smith is taking medications and gave the beneficiary a scenario with a new med for Vitamin D3, then asked him what he should do. He said it’s not on the list, so he would add “new,” and put in the information for the new medication. She asked him if he noticed that the screen changed, and he said yes. She then asked him about the text box on the previous page he did not fill in (the reason for the medication) and asked if that is something that is easy to respond to. The beneficiary said, yes, most of the time.

They moved to Other Medical Info page. The facilitator asked the beneficiary to review the screen and asked if it makes sense. He said, yes, it’s asking if someone else might have medical information. The facilitator said in our scenario, no one else does. The beneficiary noted that the examples on that page are helpful.

Then they moved to the Education and Training page. The facilitator asked the beneficiary to read the first question and then asked whether the term “last disability decision” makes sense? He said yes. So, she asked if it was a good term to use, and he said yes. She asked what that date on the screen means? He said it should be the date of the last disability determination. She then asked if he could think about examples of specialized job training or vocational education. He said any kind of course that relates to your field. She said we can continue then.

Next, they moved to Support Services screen. The facilitator asked the beneficiary to look at this question and see if the list of examples is familiar to him. He said he had heard of TTW, but that’s all. She said we would say that John has been working. He asked if he should hit yes, and she said, yes. She also said they are looking at how the information is phrased and asked him if it all makes sense. He said it does. Then the facilitator asked him to hit “add” and then move to the last piece, “types of services provided.” She asked him if he thinks this is different from what we asked on the previous page. He said he did not see where this is any different. She asked for any other comments, He had none. She asked him to move to the next page

They moved to Daily Activities screen. The facilitator asked the beneficiary to take a moment to review this page as it is a bit long, then she asked for his thoughts. He said he has no comments, other than he said someone could click on any of the issues they have. She asked what sort of information he would add in the text box of a typical day. He said that he would type in what his typical day looks like but would need to keep it down to 5000 characters. She asked what he might write there. He gave a few ideas of the types of things he does. The facilitator asked him to choose a few items from the checklist that could apply to our scenario with John Smith. Then asked him to scroll down to the next text box (that pops up only after someone chooses items). She asked for his feedback on the text box, and he said that he can use that text box to expand on the things he chose. She asked why SSA might ask this? He said to see how these things impact daily life. She asked him to continue on.

Then they moved to Work screen. The facilitator said we will say in our scenario that John worked for a bank but stopped working due to his health issues. The beneficiary hit yes and saw the short popups. She asked if this is what he expected? He said, yes, this is pretty easy.

Next, they moved to the Remarks screen. The facilitator asked if the beneficiary would put anything in here? The beneficiary said he would explain things that haven’t been covered. She asked for an example. He said he would want to expand on the impact of the disability on how he functions. Then they moved to the next screen.

They Moved to summary screen. The facilitator asked the beneficiary to review the page and explained that it will not match what we said earlier, as this screen was pre-programmed, and asked him to let her know if he has any questions or comments. He said it looks like it’s just listing everything included on the form and asked if he should just submit. She asked if he was happy with the summary. He said, yes, he would just hit submit if it looks right.

Then they Moved to Submit button. The facilitator asked the beneficiary what he would do when he sees the text box on the submitted page. He said he would print the copies of the medical release and the completed report. She asked if he could see how to do that, and he said, yes, he would click on the links. She asked if he would print or just save, or both? He said he would print it. She asked what he would do then? He said he would hit Done and log out.

They clicked Done, and the facilitator asked if the beneficiary remembered what this page on mySSA was like before? He said that he likes that it says that he’s finished with the CDR now. She asked if he thinks he can access his completed review at this point? He tried to look for it but could not find it. She then asked if he would like to have the option to access it. He said, yes, that would be useful.

The facilitator said she had a couple question now that we finished:

She thanked the beneficiary for his feedback, and asked for his overall thoughts:

He said that it is pretty easy and that’s the most important part because SSA has many people on it, and some are not as tech-savvy, so simplicity is important.

She asked if he ever completed this type of report on paper
- he said No.
She asked if he recalled any of the questions
- he said, yes, kind of like the questions he answered when he first applied.

She asked if he would use the online version over the paper?
- He said he would prefer to do it himself online as long as there are no problems logging in

She told him that right now beneficiaries get a letter saying that the CDR is due and they receive paper forms. How would he like to be informed of the CDR?
- He said by mail

She said, if you are able to log into mySSA, and are completing the CDR online, would you still want to receive the paper forms in the mail?
- He said No

The facilitator said that was all she wanted to ask, then asked him to move to the survey and be candid in his responses as they would like to know what they need to improve. The beneficiary filled out the survey.

The facilitator thanked the advocate for his feedback and opened the floor to observers.

On observer asked about the work classes: he indicated that on the training page that it could be a place to put info about vocational classes, and that the next page has the check boxes. She asked if he took some classes, where would he want to put that info first?
- He said the page with the training.
Another observer asked: did you notice that some of the fields on the screen have a red asterisk and do you know what it means?
- He said, yes, that those need to be completed to move forward.
The same observer asked for further information on that point: would you give more information that just the required fields?
- He said he would fill in as much info as possible because benefits are important, and he would want to give more info.

The facilitator thanked the beneficiary again for giving feedback and ended the call.

NOTE: We had no observers from OMB for this session, but we did have some technical difficulties with the beneficiary’s sound, which we needed to troubleshoot.

Session from 4/6/22:
- 12:00pm Session:

The facilitator explained the process and that we are looking for feedback from the beneficiary on the new screens for the CDR process, and that this will not be the only way SSA will accept the CDR (we will still accept paper forms). The facilitator explained that we will walk through a scenario with John Smith from SSA.gov, and asked if the beneficiary has a mySSA account, the beneficiary said he just signed up for myssa.gov recently, and the thinks the process could have been simpler. The facilitator asked what he found difficult, he said they asked for information that we should already have, and he was asked to call into the office, but he has not tried to log in since.

The facilitator then asked the beneficiary to tell her where he would start to get to the CDR. He said he would go to social security statement and fact sheet option, said he will find what will qualify him, and help him fill out the file. The facilitator said the letter we send says Continuing Disability review needs to be filled out. Then the facilitator asked, “are you familiar with the term CDR?” The beneficiary said yes, he has done one on the phone. The beneficiary clicked on the Continuing Disability review tab.

The facilitator then asked the beneficiary to look at the Medical Release page. The facilitator asked him to take a quick view and let her know what he thinks. The beneficiary said it was for authorization to receive records from the people who have them. The facilitator asked what the thought would happen if he selected decline. The beneficiary said he thought he will get a decline letter for benefits, or it will take longer. He clicked on I decline and read the popup message. The facilitator asked him what he thought we were trying to ask, and he said we are telling him he can print or fill it out online, but it could take longer, and said he would agree to sign electronically. The facilitator then asked him to change the response to “agree” and look at the blue text box. The beneficiary had no comments. The facilitator asked him to hit next to move on. The facilitator asked what he thought print or saving electronically the form means, and he responded that he could save or print.

Next, they moved to the Instructions (gathering info) page, and the facilitator asked the beneficiary to review it and let her know if he has any comments. The beneficiary said it was a lot to gather in. The facilitator asked if he had any questions, he said no, it is pretty self-explanatory, and explains what we need

Then they moved to the Information About You page. The facilitator told the beneficiary he will be submitting as John Smith to make sure benefits are not interrupted, and he could use his own information or information she provides. The facilitator asked the beneficiary what he thought about the questions. The beneficiary said the questions are repetitive. The facilitator asked him why he thought SSA was asking for language, and he said so that SSA could provide an interpreter. The facilitator asked the beneficiary if he selected another language did he think SSA would provide with an interpreter to continue. He said he thinks the questions will change to the language you would need, and you would be able to continue with other pages. The beneficiary said what if you didn’t understand English? And the beneficiary said he thought it would be better if the language on the screen changed.

Then they moved to the Someone We Can Contact page. The facilitator asked, “what do you think that SSA is asking for with the contact person?” The beneficiary said SSA is asking if someone other than your doctor has information about your condition. The facilitator asked him who would he put his contact, he said he would probably put his daughter. He selected yes to continue and said he would like to enter his own details. The facilitator asked him if he had any comments about the “relation to you list,” and he said no, but noted it does not specify which family member. The beneficiary selected “no” for “can this person speak and understand English,” and the facilitator asked him why he thought we are asking the question. He said so SSA can have someone who speaks their language call them.

Next, they moved to the Medical Conditions page. The facilitator gave the scenario information to the beneficiary, but told him he could make up other conditions, and he chose to use his own conditions. The facilitator asked him what he thought about the question “what assistive devises are you currently using if any?” The beneficiary said it looks fine and completed the page.

Then they moved to the Medical Providers page. The facilitator asked the beneficiary if the question made sense, and he said yes. The facilitator asked him to read it after choosing “yes” and to let her know what he sees. He said it is asking him to review what needs review or to delete what is not needed. The beneficiary clicked on the “review medical provider” and saw the doctor’s information. The facilitator asked him to look at the “when did you last see this provider” question and asked him if he remembered when he saw his last doctor. He said yes. The facilitator asked him if he keeps records of when he sees the doctor, and he said no, but he visits his doctor quite often. The facilitator asked if he hasn’t been to the hospital listed in years what would he do, and he selected delete. The facilitator asked him what he would do if he had a new doctor, and he said he would add a new doctor, and proceeded to add a new doctors’ information, and clicked save. The facilitator asked if he saw that he was successful in adding the doctor, and he said yes, it says new and is in blue. The facilitator asked the beneficiary if he saw any information on the screen explaining why we are showing the doctor, and hospitals. The beneficiary said they are coming from when we first started disability application. The facilitator informed him that it is coming from providers from their last review, and he said it makes sense and he could just update the date he saw the doctor.

Next, they moved to Tests screen. The facilitator asked the beneficiary what the medical test page tells him, and the beneficiary said tests from which you say you are disabled from, like a stress test for a heart attack. The facilitator asked the beneficiary if John had an x ray, where would he add that? The beneficiary said he would add it under “add test.” When he clicked on “add test,” it moved them to the test detail page, and the facilitator asked him to take a look at the list there and let her know what he thinks about. The beneficiary said nothing is confusing and had no comments.

Then they moved to the Medicines screens. The facilitator asked the beneficiary if it looks similar to the medical providers page and asked if it makes sense. The beneficiary said yes. The facilitator asked him to add vitamin D3 and asked him if he is taking medication would he usually know why he is taking it? He said yes, he always asks his doctor. The facilitator asked him if this was what he expected and did he expect to provide the medicines, and he said yes.

They moved to Other Medical Info page. The facilitator asked the beneficiary to read it and asked if the question on the screen made sense. He said SSA wants to know if he provided information to anyone else, he is receiving benefits from or help, and that is also states SSA wants to know about income, or if he was or is in prison. He said SSA wants to know what his bank account is like. The facilitator said we are asking for other people who know your medical information other than your medical provider, and the beneficiary responded that the question was a little confusing. The facilitator asked him to click “no” and move on.

Then they moved to the Education and Training page. The facilitator asked the beneficiary what he thought the last disability decision is, and he said your last review or when you were approved. The facilitator asked him if it would be clear have last disability review, and he said yes. The beneficiary said SSA is asking if he has gone to school for a trade. The facilitator asked him what does “specialized job, trade or vocational training” mean to him? And he said carpenter, construction worker, etc. He selected no and continued.

Next, they moved to Support Services screen. The facilitator asked if this page is clear and if the beneficiary is familiar with TTW. He said TTW looks familiar, but he has not used it. The facilitator asked him how he thought he could learn more about the TTW program, and he said he could click on the blue TTW section. The beneficiary selected yes and added Doors as the name of plan or program. The facilitator provided what types of services, test, or evaluations to add. The beneficiary reviewed the information, and the facilitator asked what the thought about this list, does it make sense, or if he any comments. He said it makes sense. The facilitator asked him what he thought the difference is between work class and vocational classes, and he said vocational classes are when you are in a classroom, an example would be cashiering, or learning to sew, and a work class is in an enclosed environment where you are learning from books instead of hands on. The facilitator thanked him and asked him to move on.

They moved to Daily Activities screen. The facilitator asked the beneficiary to describe what he does in a typical day, and what he thought he should include. He said he should include what he can do, how long he can do, how it affects his activities, if he can do it on his own, and things like how far he can walk, etc. On the “Hobbies or interests” section, the facilitator asked him if he would put anything in that field, and he said he would select yes, and would put his hobbies there. The facilitator asked if he had any questions or comments, and he had none. The facilitator asked him to look at the “have if you ever have difficulty doing the following” section and asked if the questions looked familiar. He said yes. She asked him why he thought SSA is asking these questions, and he said to see if you need help around the house, to see what you have problems with doing things, etc., and SSA may provide help that understands they need to be patient as the person may take more time doing certain things. The facilitator asked the beneficiary why he thought SSA is asking to explain difficulties completing tasks, and he said he would expand on what he selects. The beneficiary had no comments or concerns on this page. The facilitator asked him if these were the types of question he expected, and he said yes, they will let us know if they need assistance.

They moved to Work screen. The facilitator asked the beneficiary to say “yes” here to review, and that John has worked for a few months at the bank but stopped working because of the health issue. Then she asked what he thought about this. The beneficiary said he expects to have to provide a reason for quitting, and how much he made while working if he had TTW. The facilitator asked him if he expects SSA will ask these questions later, he said he would think so.

Then they moved to the Remarks screen. The facilitator asked the beneficiary if he would put anything here, and he said he would put that he is scheduled for surgery soon and would also provide the information on where it will be done and the doctor’s information so SSA would know. He said SSA does not ask for ongoing or future information, and he believes the more thorough he is with the information he provides, the easier it is for SSA to make a decision in his favor.

Next, they moved to summary screen. The facilitator told the beneficiary this was protype and not capturing everything he put in, so he hit submit. She asked him what he would do at this point, and he said he would print his documents, and clicked on “your completed report.” She asked him whether he would print or save, and he said he would do both. The facilitator asked him to click “review medical release,” and asked if he saw the electronic signature there. She also told him that is what electronic signature looks like. She asked him if the completed report is it clear that it would be saved, and he said yes. He said he would hit done and read messages. The facilitator asked him “what do you think message would be? The beneficiary said he would see that SSA received his paperwork or read any messages they sent to him.

Then they returned to the mySSA page. The facilitator asked the beneficiary if he saw the difference between this version and the first time he viewed it, and he said yes, as there is nothing that says to continue the CDR now, telling him it has been accepted. She asked him if he thought he would be able to access the CDR now that it has been submitted, and he said no, as it said it said previously you could not go back once completed. She asked him if he would like to have access to the form after it’s completed and submitted, and he said he would like to have the ability to access after it was submitted in case he loses the form he has.

The facilitator then asked the beneficiary for his overall thoughts:

He said he likes it, and it is simple if you take time to read, states it’s a lot to read, but explains what is going on if you take your time to read. He likes the ease of navigation and the design.

She asked if there was anything we could do better?
- He said he states he does not like repetitive questions for language, reading and writing.

She asked if he has completed it on paper and whether he has ever called SSA for help.
- He said when he first did the review, he did it on the phone, and it took about 10-15 minutes, and that they didn’t ask him all these questions. He said they asked if he still had condition he had, medications he was taking, and if he had any new conditions, if address was still correct, and any new education, if he was recently incarcerated. He said didn’t take too long maybe 10-15 minutes.

The facilitator asked if he would complete the CDR online or call SSA
- He said he would rather call SSA to have it done over the phone, because there are too many questions.

The facilitator also asked how he would like to be notified when a review is coming (states online version will not be available until early next year).
- The beneficiary said he would like to be notified by email, he said the mail takes too long, and the last time he received a letter by mail, it was after his scheduled review.

The facilitator asked if he expects SSA to still send the paper version of what we just completed.
- The beneficiary said no he does not want a blank version of the form she just did online.

The facilitator then asked the beneficiary to complete a satisfaction survey.

The beneficiary responded to the survey

The facilitator thanked the beneficiary for his feedback and opened the floor to observers.

An observer asked: if he remembered when he added a new doctor, would he take the effort of finding correct addresses, or would he put in an approximate address (name of street).
- He said he would put in the name of institution if he did not know the exact address.

The same observer followed up and asked him if he would take any effort to find an address
- He said yes.
Another observer asked him: on the medication page where there was a list of medications, there was a field for “reason for taking medication,” do you know the reason you are taking your medications, and would you put that explanation in the reason field?
- He said yes.

The facilitator thanked the beneficiary again and ended the session.

NOTE: We had no observers from OMB.

Session from 4/7/22:
- 2:30pm Session:

We waited in Zoom for 20 minutes, but the beneficiary did not attend. Per the contractor who scheduled the appointment, the beneficiary had technical difficulties that kept her from today’s session. They cannot reschedule her due to the time, but they are looking to see if they can find someone else for a session tomorrow (Friday). We ended the call at 2:50pm.

NOTE: We had no observers from OMB for this session

Session from 4/8/22:
- 2:30pm Session:

We waited in Zoom for 15 minutes, but the beneficiary did not attend. Per the contractor who scheduled the appointment, they could not reach the beneficiary. They cannot reschedule him as we have completed the week of testing. We ended the call at 2:45 PM.

NOTE: We had no observers from OMB for this session.

Round 2 Usability Testing Overall SSA Evaluation:

Based on a study with 5 disabled beneficiaries and 5 disability advocates from March 29 to April 7, 2022. Two out of 5 advocates assisted with evaluating this application in the fall of 2021. Summary of key findings is followed by detailed, page-by-page, findings.

The objective of this round of usability testing was to retest the i454 web application to assess how well it supports its intended users given the changes made using feedback from the first round of testing, public comments, research with DDS and FO SMEs, and the proposed SSA-454 paper form as of March 24, 2022.

Summary of Key Findings

The overall satisfaction of beneficiaries with the form has improved since the 1^st round of testing
Both the beneficiaries and disability advocates felt that propagating details of medical providers from either the initial application for benefits or the most recent medical CDR was a major improvement to this application
Both user groups found propagating the date of last disability decision beneficial
Most users understood the purpose of the Medical Release Form or 827 and felt that the instructions on submitting the entire CDR either electronically or on paper are clear
Disability advocates reiterated that responses to Daily Activities are used to deny benefits with subsequent Medical CDR appeals. They also reiterated redundancy of this section with the Function Report, subsequently requested by DDS for most cases
Most participants had difficulties understanding the English proficiency questions, expecting the form itself to change to the language of their preference
Both advocates and beneficiaries questioned how one can complete this form if they don’t have the command of English language. Some expected they would not be able to proceed with completing the form if their responses indicate lack of proficiency and/or literacy
Most users expected to be able to access their completed medical CDRs later
Users said they prefer not to receive the blank paper SSA-454 and SSA-827 forms by mail if they chose to use i454
While adoption of mySSA services continues to grow, both advocates and beneficiaries expressed that accessing the portal is somewhat difficult, which directly impacts access to i454

Accessing Medical CDRs from mySSA Homepage

While adoption of mySSA services continues to grow, both advocates and beneficiaries expressed that accessing the portal is somewhat difficult, which directly impacts access to i454
Most participants found the card to access the Medical CDR without problems
- Several mentioned the “Action Required” badge as helpful in finding the link. One participant said, "it seems like the most urgent" thing on the page
- One advocate suggested changing the wording from CDR to 'your disability case'
- Another suggested the card could be more prominent

Medical Release Form or 827

Most users understood the purpose of the Medical Release Form or 827 and felt that the instructions on submitting the entire CDR either electronically or on paper are clear:
- “It is as clear as it could be. It is a bit long for people to read, but it is understandable since policy need to show this language”
- "that's probably about as clear as you can make it while still keeping it informative... I think you have policy and legal rules about how much you can say…"
- Two beneficiaries were concerned about losing their benefits if were to select decline
Some advocates suggested to consider refining the language to make it more accessible:
- For instance, the 2^nd paragraph could be more explicit, 'by signing the form, you give us permission to request records'
- It may be clearer to say “You can do it here electronically right now or you'll need to get paper and fill this out”
- "Decline might not ring a bell to some people with limited vocabulary"

Instructions

Most participants thought the instructions made sense and were clear:
- “Nothing confusing to me on this page, self-explanatory, just telling me what they need”
- “It is pretty cut and dry”
One of the beneficiaries thought that it was a lot of information to manage: "Oh, my goodness! I guess it is a lot of information you have to gather”
One advocate thought the instructions should be more explicit regarding other contacts information: “Make clear you are not asking for medical source information”

Information about You/Language Proficiency and Literacy Questions

Unlike the prior round of testing, this time all language questions were on the same page, leading to fewer concerns that the same questions are being asked twice in separate sections
Most participants had difficulties understanding the English proficiency questions, expecting the form itself to change to the language of their preference
Still, several participants noticed an explanation about the interpreter services for non-English speakers
Both advocates and beneficiaries questioned how one can complete this form if they don’t have the command of English language. Some expected they would not be able to proceed with completing the form if their responses indicate lack of proficiency and/or literacy
One advocate asked whether non-English speakers will be directed to completing the i454 given it is available only in English

Someone We Can Contact

Unlike the first round of testing, most participants found this page easy to understand given the changes to clarify the page title and instructions
Most understood SSA was asking for someone who can corroborate a medical condition other than a doctor, it was better explained based on earlier feedback
Several comments were in reference to the drop-down relationship list:
- Some thought it might be helpful to know the actual relationship between family members, instead of just generic 'family member'
- At least one advocate suggested “case manager” be added to the drop-down list of contacts, which was suggested in the prior round of testing as well

Medical Conditions

All users were able to accurately document medical conditions, one condition per entry. This improvement was clearly due to presenting multiple textboxes at once (three in this case) with the ability to add more if needed.
Most users were content with adding height and weight in this round of testing, most likely due to removing “without the shoes” phrase. Several users recognized that providing height and weight is optional

Assistive Devices

All users were comfortable selecting assistive devices from the list
Advocates expressed reservations about the checklist as it does not reflect on new regulations requiring specifics on frequency of use and if prescribed by a doctor:
- “Under the new regulations walker and wheelchair matter a great deal. Before it just asked if a person needs assistive devices, but now regulations ask … if device was prescribed by a doctor, how much it is used, etc.”
- "It matters if you have one or two canes… As a beneficiary, I might not be thinking much to make a big difference between one cane or two canes, and I will just click the check box. But for advocates/DDS that information is meaningful. Maybe when this information gets to DDS, it could be flagged somehow, so they can pay close attention to those answers"
- “Power scooter or wheelchair (one hand or two hand device)?”
- “Hearing aid: does it include a hearing implant?”

Medical Providers

Both the beneficiaries and disability advocates felt that propagating details of medical providers from either the initial application for benefits or the most recent medical CDR was a major improvement to this application
Only two participants were unsure where the providers data originated from, but it did not deter them from completing the task
Users suggested some language refinements to clarify that data can be propagated from last review as well as initial application
Participants understood they need to review providers for accuracy and liked the badges indicating providers needing review, already reviewed ones as well as just added
Only one beneficiary was hesitant deleting a provider they have not seen in two years (according to the scenario):
- “If I delete this medical center, will it be deleted from my file completely? If so, how I can find that information in case I need to reference it for something else?”
Advocates that participated in the first round of testing really liked getting away from separate Doctors and Hospitals entries, which was supported by the ease of adding medical providers observed
Collecting only the date last seen was clearly an improvement
Yet just as in the prior round of testing, majority of participants expressed it is hard to remember dates of appointments
- "I have a problem thinking about exact month and year"
- “People might not really remember if they saw that doctor 10 month ago, and people will skip that part and will not add new or review old record”
Advocates suggested that the date last seen is not relevant nor critical given the 12-month window of reporting
- “To me, if somebody says that they have seen a provider in the last 12 month, the records will be requested. Does this date represent the starting date when records should be requested from?"
One advocate and one beneficiary suggested to add a description of conditions treated for new providers

Tests

Participants liked the Test section that is no longer asking for the date of the test
Participants liked that ‘Ordered by’ includes names of previously added providers while allowing to enter details of a new one if needed:
- “Sometimes people get confused because for tests they go to radiology clinic but the doctor who ordered tests is not at the radiology clinic. And I think this list tells what you want to know, doctor's name instead of radiology clinic”
Advocates liked the addition of Psychological/IQ Test
Two advocates questioned why Blood Test (not HIV) is listed like that, separately from HIV:
- “Why do we need to mention HIV on blood test? it is not an issue nowadays. I know it came from other forms”

Medicines

After completing reviewing Medical Providers, all users were familiar and comfortable reviewing propagated medicines
However, users questioned again the need for collecting medicines as the list can be potentially very long while spelling of medications is challenging:
- “Medicines could be a long list and hard to enter, because of long word spelling and small print hard to read. I do not know what can be done to make it shorter. If DDS knows that the beneficiary has cancer, does it really matter what kind of chemotherapy drug they are taking? If the list of medicines will help people to continue get benefits, then they should enter them, but if not … maybe it could be skipped and save beneficiaries' time”
All advocates felt that documenting side effects is important since it is not going to be found in medical records. This echoes suggestions from prior round of testing
Four (4) out of 5 beneficiaries suggested they were aware of reasons for medications. Two advocates felt the question could be clearer

Other Medical Information

Three (3) out 5 beneficiaries were confused by the question, with two of them believing the question is to establish whether they receive benefits from other organizations as opposed to identifying who else has their medical records
Advocates agreed that the question is unclear and suggested to revise it: "Who else might have medical information about you? What information they might have?"
One advocate pointed out that asking for the next scheduled appointment here is inconsistent with Medical Providers where it is not asked. Another felt that dates are not necessary here.
One advocate noted that beneficiaries may not know the contact person and expressed this field should not be required
One advocate stated that asking for Reason for Contact when someone is incarcerated does not make sense, which echoes comments from the first round of testing

Education and Training

Both user groups found propagating the date of last disability decision beneficial
Three (3) out of 5 beneficiaries as well one advocate were somewhat confused by the term "last disability decision" suggesting "last disability review" as a more clear option. Two other beneficiaries were fine with the "last disability decision"
Advocates and beneficiaries alike, could not tell the difference between vocational training and vocational rehabilitation under Support Services:
- "Vocational training and vocational rehabilitation may be not the same thing, but beneficiaries might not realize it"
- "I think the word 'vocational' in training description throws me off. The same word is used on Support Services." One advocate suggested to combine vocational training and support services
One attorney anticipated to see here information currently hosted under Support Services:
- "A lot of my clients who go back to school do so with the support of a disabled students’ program"
One advocate was concerned that just because someone is enrolled in GED does not mean they are successful. She was wondering whether DDS cares how well the beneficiary is doing in those programs, or they just care about completion date and if you really completed the program

Support Services

Users liked the links to specific support programs like Ticket to Work
Both advocates and beneficiaries could not differentiate between work classes and specialized job training: "I would simply delete 'work classes' from this list, leave 'work evaluations' in the list though and add 'medical evaluations/examinations', since you already have 'psychological/IQ test' since those support services do send beneficiaries for medical and psychological evaluations"
One advocate questioned why Psychological/IQ Test was listed here while others did not comment
One advocate noted: "Type of services’ choices is not very comprehensive list, if we add Individualized Education Plan (IEP), it might include more tests. In IEP, the type of support varies. They do testing for learning disability and other impairments"
Also, "IEP program might not have individual coach or instructor." Further, "IEP or accommodations at school might not be thought of as they are educational programs"
Users found the main question confusing: "I find this a little confusing. The question asks about services helping them go to work, but clients may not think of an IEP as helping them with work, but only with education"

Daily Activities

Daily Activities changes since the last round of testing were minimal, repeat concerns were expressed by both user groups
Several advocates reiterated in this round of testing that Daily Activities section is redundant with the Function Report and not needed: "Function report will provide you with fuller and better information and better way to collect that information"
Advocates feared that it is impossible to capture information needed unless specific questions are asked as beneficiaries tend to write about their capacity which is not what DDS needs
Disability advocates reiterated that responses to Daily Activities are used to deny benefits with subsequent Medical CDR appeals:
- One lawyer who commonly handles denials said, "never once, in over a decade of doing this, I've seen it [Daily Activities] used in a way that's beneficial for a claimant. These are decisions that we routinely get overturned." They don't have an accurate understanding of their own abilities compared to what is being looked for here, and the judge relies on what they say against all other evidence and denies a claim. These are often overturned. This gets used in harmful ways and isn't supported by the other evidence
- "I really don't like the question on hobbies and interests. This is a problematic question that gives support for an unfavorable decision. I've never seen it used it any other way. I have strong feelings about this: I don't think it is useful, it is kind of demeaning, and problematic. Having a hobby or interest without any contextualization of your daily life or how well you are able to engage in this hobby is problematic. I've only seen it used in negative ways"
- Not sufficient context for difficulties doing various activities "It isn't clear enough that these must be done independently, without help. It doesn't speak to the quality of the task"
The list of activities is too long and could be grouped rather than listing 20 items
Beneficiaries felt this page is focused on what you can versus cannot do because of the disability

Work

All users expected to be asked for other details such as dates of employment, employer name, wages if more $400, some explanations about work attempt even if unsuccessful, part-time or full-time. Since this was not the case, they expected to receive another form later
At least one attorney suggested to link to SSA-820 and/or SSA-821 since these forms will have to be submitting (both are available online) for those worked

Remarks

Some said they would not enter anything in the Remarks, others may reiterate their conditions. One bene suggested he would add an upcoming surgery in remarks, another would add a new condition if needed.

Summary and Review

Beneficiaries reportedly would like to have the ability to print, save as well as access their completed CDRs later
One attorney asked if it would be possible to combine the Report and Signed SSA-827 into a single receipt
Attorneys mentioned challenges requesting completed forms from SSA; some were concerned that beneficiaries may not print or save initially until denied benefits and seeking legal counsel

Additional Feedback

Most users mentioned email as their preferred way of notification about an upcoming review. Overall, multiple ways to reach out were suggested including notifications to mySSA, letters as well as automated calls
Users said they prefer not to receive the blank paper SSA-454 and SSA-827 forms by mail if they chose to use i454. While not all users were asked the question, all that were asked expressed this preference
While adoption of mySSA services continues to grow, both advocates and beneficiaries expressed that accessing the portal is somewhat difficult, which directly impacts access to i454
Advocates asked when i454 will be available to them
Advocates expressed concerns that they are not notified about an upcoming Medical CDR even for people they have been representing/providing legal counsel in the past

Conclusion

Usability Testing Summary from the paper SSA-454 Sessions Held from May 3, 2022 - May 10, 2022.

This report is based on a usability testing of the paper SSA-454, conducted by the UXG team with 5 disabled beneficiaries and 2 disability advocates from May 3-11, 2022. Our recommendation has been to recruit 5-7 participants from each target audience; however, we had recruiting challenges with both beneficiaries and advocates. Nonetheless, the qualitative data collected in this study are invaluable to our efforts to identify improvements to the paper SSA-454.

The objective of this testing was to evaluate how well the revised paper SSA-454 supports its intended users. Test participants had to explain their understanding of the questions and content to help assess comprehension, readability, and print design of the form.

Summary of Key Findings

Advocates, familiar with the current version of this form, recognized streamlined details of medical providers/healthcare professionals as an improvement
Two out of five beneficiaries had trouble understanding the purpose of the form
Beneficiaries were unable to locate the date of last medical disability decision, nor could they understand the term
Several beneficiaries felt that certain sections of the form should be completed by their physician
Users described the form as lengthy and overwhelming by the time they got to Other Medical Information but overall were more concerned with clarity than number of pages
Users were concerned with limited space for medical conditions, providers, and medicines and suggested to increase the number of entries, to preserve proper formatting/level of details that would not be possible if documented under Remarks
Users found questions within Education, Training, and Support Services both confusing and redundant
Users found the order of sections problematic, such as moving Other Medical Information further away from Medical Information and moving Who Is Completing This Report to the end of the form
The purpose and the placement of English proficiency and literacy questions were problematic
Beneficiaries were unclear on the purpose of Daily Activities’ questions. Concerns of redundancy with other SSA forms and sense of mistrust were expressed
Both advocates questioned instructions for completing certain sections based on age
Users pointed out that the three dates collected in Other Medical Information were not only confusing but also did not align with a single ‘date last seen’ for medical providers
The overall satisfaction of beneficiaries with the revised SSA-454 form resulted in a score of 70.4%

Instructions

Overall, the instructions were clear and helpful
One advocate expressed a concern with using a general SSA phone number to call with questions: her experience suggests that Tele Service Center staff is likely to direct callers to local offices for questions

Date of Last Decision

Beneficiaries were unable to locate the date of their last disability decision largely because it was located under For SSA Use Only. Advocates suggested to consider another way of providing this date, potentially in a cover letter that is part of the mailer
- I did not pay this any attention because it says ‘For SSA use only’
- People will not be looking at the box 'For SSA only", so the date of last decision people probably will not notice or even attach it to themselves
- Do not write in this box, this is going to be scanned. Nobody hand check the boxes, why do we need this section?

Beneficiaries had major issues relating to the date of last disability decision: most of them could recall the date when they were approved for benefits but not subsequent reviews

Advocates asked if the ‘For SSA use only’ section is needed on this form

Medical Information: Height and Weight

At least one advocate suggested to list medical conditions first and then height and weight because conditions are more important. This aligns with previous reports by DDS
One advocate noted that if not USA as not pertinent:

Note: Users did not realize the introductory paragraph (shown as bolded above) pertains to the entire section and not height, weight, and conditions. One advocate when probed suggested to remove the intro paragraph: it is inaccurate to refer to data collected as general medical information; the second sentence is applicable to anything in this form and provides no value.

Medical Information: Conditions, Providers, Tests, and Medicines

Advocates, familiar with the current version of this form, recognized streamlined details of medical providers/healthcare professionals as an improvement
Users were concerned with limited space for medical conditions, providers, and medicines and suggested to increase the number of entries, to preserve proper formatting/level of details that would not be possible if documented under Remarks
- I don't think it will be enough space, also spacing between lines is tight, and person with disability might have a problem to write in small letters. I would list just name of my disease and leave it like that and will not put any symptoms. If I run out of space, I would put an arrow and write on opposite side or get a blank sheet of paper
Several beneficiaries felt that certain sections of the form should be completed by their physician
- In my opinion this form should be filled by physician. Some of the questions like medical conditions how can I answer that? This form could be like a catch when they try you to say something that doctor cannot confirm, sticky point. In my case, I see doctors that assigned by SSA, and their evaluation makes the case, not my own thinking
- I know the medical evaluation that come from doctors makes a difference, not my opinion

Medical Information: Assistive Devices

Beneficiaries felt that the list of assistive devices was clear
One advocate found the terms sometimes and always could be difficult to assess:
- Someone with a broken hip may not need a wheelchair for getting around the house but always need it when going out. So, what to choose – sometimes or always?

Date Last Seen

While limiting dates to Date Last Seen for providers was viewed as positive, advocates questioned having the date if there is no impact on the process of requesting records:
- I feel the medical records should be requested through the present anyway, and the form should match the procedures, and if the procedure to ask for the records ending on the certain date, then it does make sense to ask the date question, but if the procedure asks for 12 months through the present, then why to ask the last seen date question?

Other Medical Information

Some beneficiaries did not recognize the difference between this section and the Medical Provider section of the form:

Those who did understand the purpose of this section had to re-read the instructions very carefully
One advocate predicted that most beneficiaries will just skip over, which was echoed by a beneficiary’s response:
- I guess I will skip to section 7
Users pointed out that the three dates collected in Other Medical Information were not only confusing but also did not align with a single ‘date last seen’ for medical providers:
- What do you mean by date of first, last, and next contact? This just seems crazy!
- Dates I will not be able to remember or find, the only way to find them out is to talk to those individuals from programs

Support Services, Education, Training

Users found questions within Education, Training, and Support Services both confusing and redundant
One advocate expressed that the list of services under Support Services is limited:
- A lot of services are not there - physical therapy, speech therapy, emotional regulation etc. Honestly, I will almost leave this question blank. It is a random list. Why not let people provide own test/services? You have a mixture of tests and then work classes - it is a service, the only service here, where you could have much more services, like speech therapy, psych therapy, occupational therapy, physical therapy, or service like when somebody reads your test to you in class, because you cannot read, or service like alternative communication device, or like IEP will allow you to leave classroom because you feel overwhelmed, or do not attend first period, because you always feel nauseous in the morning. So, it feels like a mish mash of things you might get from work rehab or school. There are so many things that are not included here. Just have a blank field and let people write what they want

All users had difficulties explaining how work classes differ from specialized training
- Work classes, I do not know what it means. I would guess, could be something for industry like white- or blue-collar jobs
- Work classes sound remarkably similar to training
Several users found the Education question too general and, therefore, confusing
The dates in all three sections were either confusing or difficult to obtain
- Expected completion date is the date when I stopped. I would put both dates - the completion date and the date I stopped the program

Work

Beneficiaries were able to complete work details:
- Both advocates and some beneficiaries expected additional questions if someone is currently working
- One advocate asked if restricting responses (in this section) to age 14 or older was for a policy reason:
  - Conceivably, someone younger than 14 can be working like be a child actor

Both advocates suggested to reverse the second and third questions for a more meaningful flow:
- The order of work questions for me logically would be: 1) have you worked, 2) what kind of work you do and then 3) are you currently working

Suggested Ages for Completing Certain Sections

Under Other Medical Information, Advocates questioned restricting this section to beneficiaries of the age 18 and older:
- Why other medical information is only for age 18 and up? So younger kid who was hit by a bus cannot have attorney? Kids also can get social services or be incarcerated. Why there is such restriction? What about child welfare agency, like kids in foster care? Sure, not many kids will receive workers’ comp, but the rest of places easily can be used by kids
Under Support Services, one advocate expressed reservations with the usefulness of the question for age 18 youth:
- I don't think this question will be useful for age 18. They are just in high school. Yes, they might be in special ed, but still, they study PE and Science. For transition age youth who fill this form it could be confusing

Note: during prior user research, DDS staff expressed that Sections Other Medical Information, Support Services and Education and Training do not depend on age 18 for Title II cases where same rules apply for adults and children.

English Proficiency and Literacy

Beneficiaries struggled to understand why language related questions were split between the two sections. They felt the questions were redundant:

Advocates expressed concerns with validity of responses if completed by someone who does not understand English. Moving the section Who is Completing This Report to the very end only reenforced the issue:
- If the beneficiary cannot read and understand English, they will not be able to even check 'yes/no' for this question. Probably, the bene will have to find somebody who understands English to help with filling out the form
- It's a little silly. If you don't understand English, nothing you completed on this form would be valid
- These questions are pretty deep in the form to figure out if somebody cannot read. Ask them at the top of the form
Users reiterated that beneficiaries’ English proficiency and literacy should be already known:
- This should be documented in their already as they are answering this same question when they first file. These questions are while simple but not necessary at this state

Advocates challenged how the questions are asked:
- If you cannot read, would SSA provide an interpreter? The section is incoherent. Why speak and understand and separately read and understand?
- I can read a simple message but still can be functionally illiterate. And you are not really capturing that
- Simple message: are you asking about their motor skills? Dexterity or literacy? Voice to text is not writing

Daily Activities

Some beneficiaries thought that this section was redundant with questions asked in other forms:
- I was already diagnosed with disability. I had reviews every year from the program I am receiving [benefits from]. They are doing functional screening of me every year

Others were confused about the question on daily activities: o Describe typical day... I have no idea why SSA asks that. I don't see what the importance of that question could be
- Not sure why it is asked or why it might be important
- I do not know what they trying to find out

One beneficiary questioned asking information about hobbies:
- I am surprised to see hobbies and do not understand how they related and helpful to disability. I could be disabled and love poetry, so how it is related or benefit to my disability case?

Who Is Completing This Report

Most users had trouble identifying the person in 2A
Users reported that moving Who Is Completing This Report to the end of the form is problematic (see language considerations above) and should be asked early in the form, for instance under Someone We Can Contact
- Now you are asking that?! I would put it at the top. People may not realize it is even here!

Satisfaction Survey

The overall satisfaction of beneficiaries using the revised SSA-454 form resulted in a score of 70.4% (see Appendix A), which is considered average

Appendix A. Beneficiary Satisfaction Survey of the Revised Paper SSA-454 Form

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	Addendum to the Supporting Statement for Form SSA-3462
Author	177717
File Modified	0000-00-00
File Created	2023-08-01

Addendum - 0072

Addendum - 0072 (Final).docx

Continuing Disability Review Report

Addendum - 0072

OMB: 0960-0072

Table of Contents

Background

Revisions to the Information Collection

Terms of Clearance

Public Comments on the Information Collection

60-Day Comment Period Federal Register Notice (FRN):

Overall Comments on the Time Estimate and Length of the SSA-454-BK

Overall Comments on the Complexity of the CDR Process and Stress on the Respondents

Comments on the Frequency of CDRs

Comments on Improving Form SSA-454-BK

Comments Regarding Specific Revisions to Form SSA-454-BK

Comments Suggesting SSA Complete a Behavioral Audit on the CDR Process and Form

Comments on the new i454 Internet Modality

30-Day Comment Period Federal Register Notice (FRN):

Usability Testing Notes from the Round 1 i454 Sessions Held from 10/18/21 – 10/27/21

Usability Testing with SSI and SSDI Recipients (10/18/21 – 10/27/21):

Usability Testing with Advocacy Group Members (10/25/21 – 10/27/22):

Round 1 Usability Testing Overall SSA Evaluation:

Usability Testing Notes from the Round 2 i454 Sessions Held from 3/29/22 – 4/8/22

Usability Testing with Advocacy Group Members (3/29/22 – 4/1/22):

Usability Testing with SSI and SSDI Recipients (4/4/22 – 4/8/22):

Round 2 Usability Testing Overall SSA Evaluation: