OMB Package: 60-day Comments
Testing Experience and Functional Tools Demonstration: Personal Health Record User Survey Instrument
Prepared for
Barbara J. Holt, PHD
Health Insurance Specialist
Division of Community Systems Transformation
Disabled & Elderly Health Programs Group
Centers for Medicare & Medicaid Services
(410) 786-2079
Prepared by
The Lewin Group
On June 8, 2016 the 60-day Federal Register Notice (FRN) was posted for the TEFT Personal Health Record (PHR) User Survey. The comment period closed on August 12, 2016.
No comments were received through the FRN via monitoring FDMS (fdms.gov).
CMS and contractor, The Lewin Group, received comments from two TEFT states (Georgia and Maryland) via direct email. The response to comments is outlined below.
Comment 1: If we use support planners to administer the survey, this question is N/A. The answer will always be “With help from a case or care manager” (Maryland)
Response: If the state uses support planners to help facilitate the survey, this question will be pre-checked for the state.
CMS Action: No survey change needed.
Comment 2: This question [Question #3] should come before #2 if possible without disrupting the skip patterns. If they have never heard of it (as indicated by Q2), this question might be repetitive, which could lead to frustrated respondents (Maryland)
Response: Agreed.
CMS Action: Question #3 is now Question #2 with a skip pattern that is appropriate for the response.
Comment 3: Answer choices not applicable to planned system capabilities at initial launch (Maryland) – applicable to Questions #4, 5, 6, 9, and 11.
Response: Since states are fielding different Personal Health Record (PHR) systems, the generic answer categories will be deleted, as needed, to represent the offerings of that state’s PHR. The question will remain the same.
CMS Action: Answer categories not relevant to a state’s PHR will be deleted before survey fielding.
Comment 4: We won’t allow updating of information at this time [Question #9] (Maryland)
Response: Since states are fielding different Personal Health Record (PHR) systems, the question wording of “do you view or update in…” will be modified to “do you view in..” for states that do not allow PHR updating.
CMS Action: The question wording of “do you view or update in…” will be modified to “do you view in..” before survey fielding.
Comment 5: Tailored for Each User Type: The PHR is going to be used by multiple roles (clients, family members, and service providers). As such the survey should accommodate this through the use of skip patterns and customized questions for each role. (Georgia)
Response: The intent of the survey is to focus on the Medicaid Long-term Services and Supports (LTSS) Waiver beneficiary. While we recognize that the beneficiary may receive help or that a caregiver may fill out the survey on their behalf, we will not be developing a survey for each user type.
CMS Action: Question #1 has been modified to reflect the Medicaid beneficiary receiving help from someone in completing the survey or that someone else is completing the survey.
Comment 6: Judging Impact: Consider doing a pre- and post- survey to better judge impact of the PHR. A baseline measure will better enable us to gauge preliminary interest and outcomes. (Georgia)
Response: The survey will be fielded three (3) months after someone is offered the state’s PHR. There are no plans for CMS to field a pre-PHR use survey.
CMS Action: None
Comment 7: Technology Experience: Because users’ technology experience impacts their experience of the PHR, it is important to have a questions asking about this. Attached is a validated 4 question survey. I’m not aware of any shorter surveys. (Georgia)
Response: The CMS team reviewed the four (4) question survey offered and decided not to use the survey as it was because (1) 4 additional questions were too many and (2) the likert rating is not advised for fielding a survey to some Medicaid beneficiary populations (e.g., those with ID/DD or TBI).
CMS Action: CMS used the four (4) question survey offered as the basis for a new question on technology use [Question #24].
Comment 8: Role: In relation to the Tailored for Each User Type comment (see comment #5), please include a question related to user role. (Georgia)
Response: CMS will not be tailoring the survey for each user type.
CMS Action: Question #1 has been modified to reflect the Medicaid beneficiary receiving help from someone in completing the survey or that someone else is completing the survey. Additionally, Question #26 has been added to receive qualitative feedback specifically from other user types.
Comment 9: Comment on Question #1: This question has a long set of responses. If the purpose of this question is to understand influence, then simply ask if they had help or not. If they did have help, then consider asking this longer question. Perhaps a branched question would be more usable. (Georgia)
Response: Agreed.
CMS Action: Question #1 has been modified to reflect the Medicaid beneficiary receiving help from someone in completing the survey or that someone else is completing the survey.
Comment 10: Comment on Question #3: Consider rewording to “Have you used…”. This question may not be a good measure of “usage” as our PHR could involve using a paper copy, leveraging texting functionality, or other features that do not require logging onto a website and may not be considered as “use” by the user. Consider breaking this down by types of interactions that may constitute use of the PHR. (Georgia)
Response: The intent of the question is to determine active use (given the answer categories).
CMS Action: The question remains “do you” rather than “have you”. CMS clarified language for “use” as “view or update” and provided an example of what that includes for a PHR.
Comment 11: Comment on Questions #4 and 5: This question [Question #4] gets at motivation. Some responses seem abstract. See additional comments/questions to consider below regarding using the Importance/Satisfaction section. It seems like questions 4 and 5 should align in some ways or be replaced with the Importance/Satisfaction method described in the section below. Some of the questions in this are rating user experience aspects of the PHR. Consider having separate questions regarding “barriers to use” and “user experience” instead of bundling all of this together. (Georgia)
Response: CMS agrees with combining questions #4 and #5 and using some of the categories in the provided Importance/Satisfaction method document.
CMS Action: CMS combined questions #4 and #5. CMS will not use the Importance/Satisfaction method but did adopt some of the answer categories provided in that document. CMS will continue to use agree/disagree answer categories rather than likert scale options.
Comment 12: Comment on Question #7: Why does this question matter? Are you trying to get at understanding public/private use and security concerns related to access to health information? Consider removing this question. (Georgia)
Response: The intent of this question is to understand where an individual most frequently uses the PHR and whether they receive help in using it.
CMS Action: CMS split this question into two questions to separately address where someone uses the PHR and whether they receive help.
Comment 13: Comment on Question #8: See comment on Question #3. (Georgia)
Response: The question will be modified similarly to Question #3.
CMS Action: Question #8 was updated to modify “use” to “view or update”. What “use” constitutes will be written out in the introductory text of the survey.
Comment 14: Comment on Question #9: Our design includes additional information. See the list below. Also, consider adding a questions regarding “What kinds of information do you communicate to your service providers?” and “What kinds of information do you receive?” (Georgia)
Additional View or Update Information:
Care Team contact information
Care Team availability
Educational resources about my conditions
Information Received
Reminders about upcoming doctor appointments
Reminders about upcoming home visits
Reminders about eligibility
Reminders about medication refills
Information to Communicate
Allow my caregiver to get updates about how my day is going
Allow my caregiver to get updates about my health status (e.g. doctor visits)
Allow my caregiver to view information about who I am and what I care about
Allow my caregiver to view information about possible health concerns
Allow me to easily communicate issues with my support
Response: Agreed.
CMS Action: CMS modified Question #9 to focus on viewing or updating social services information, Question #10 to viewing or updating health information, Question #11 to focus on the information someone receives from their PHR (if applicable), and Question #12 to focus on information someone gives access to another from their PHR (if applicable). Please note that if these types of offerings are not made available by a state (e.g., updating or receiving information from a PHR) the questions will not be asked in that state.
Comment 15: Comment on Questions #10 and 11: There are two ways information can be communicated to a care team member: 1) The client proactively shares that information, 2) The client grants access to that information. The GA PHR primarily uses the second option. Consider rewording to “I have shared (or given access to) information…” (Georgia)
Response: Agreed.
CMS Action: These are now Questions #13 and 14. Any reference to “shared” now states, “I have shared (or given access to)…”.
Comment 16: Comment on Question #13 (now question #15): It’s difficult to know if the PHR was not used because of a poor user experience or because it did not accomplish the goals of the user. The Importance/Satisfaction method [referenced in Comment 11] may address this better. (Georgia)
Response: CMS will modify this question to address answer categories in the Importance/Satisfaction method, but not to adopt that method.
CMS Action: CMS will not use the Importance/Satisfaction method but did adopt some of the answer categories provided in that document. CMS will continue to use agree/disagree answer categories rather than likert scale options.
Comment 17: Comment on Question #14 (now question #16): Consider removing this question. The Importance/Satisfaction method described [in Comment #11] may address this better. (Georgia)
Response: The intent of this question is to gauge interest in using a PHR.
CMS Action: No change. Since the Importance/Satisfaction method is not being adopted for this survey, the question remains the same.
Comment 18: Recommendations: Consider including questions such as “Would you like to continue using the PHR if it were available?” and “How likely are you to recommend the PHR to a friend?”. Again, you may want to break this down into the various functions of the tool (text messaging, etc). (Georgia)
Response: Added.
CMS Action: Response options of “I would recommend the [Insert state-specific PHR name] to a friend or family member.” And “I would like to continue using the [Insert state-specific PHR name].” were added to Question #4.
Comment 19: Importance/Satisfaction Method [referenced in Comment #11] The output of this methods allows you to highlight features that are important but did not result in high satisfaction scores. (Georgia)
Response: This method will not be adopted.
CMS Action: This method was not adopted for the survey to maintain agree/disagree response categories, avoid asking duplicative questions which may confuse the respondents, and to maintain a shorter survey length.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | OMB 60 Day Comments |
| Subject | TEFT PHR User Survey |
| Author | The Lewin Group |
| File Modified | 0000-00-00 |
| File Created | 2021-01-23 |