Collection of Data to Support Training Decay Selection for Medical Product Usability Validation Testing (CDER)

Training Decay Selection for Medical Product Human Factors Validation Testing

Moderator’s Script for Study Sessions

UserWiseFDA03, Rev 02

OMB Control No.: 0910-0847

Expiration Date: 12/31/2022

Training Decay Human Factors Study

Moderator’s Script for Study Sessions

IMPORTANT: Note that text in italics or bold should NOT be read aloud to participant.

1. Untrained Users - Introductory Script

1. Key Reminders for Untrained Participants

• Make sure the section regarding potential risks has been read out loud to the participant.

• Make sure that the participant has signed the ICF.

2. Introduction for Untrained Participants

• Hello, my name is ______________. I am the moderator, and I’ll be walking you through this session, and _______________ is the observer.

• First off, we want to thank you for your time and participation in this usability study.

• Before we get started, do you have any questions about the informed consent form?

• The purpose of this session is to evaluate an insulin pump to identify whether you can use it safely in a simulated use environment.

• The combined results of all of the participants in this study will be published, but your identity will always remain anonymous.

• If you feel unwell or wish to end the session at any point for any reason, please let me know.

• For purpose of this evaluation, we request that you turn off your phone. We appreciate if you do not use your phone throughout the session.

• For purpose of the session today, I want you to imagine that you are a caregiver. The manikin, here, Sam (point to manikin) will represent your patient, which may be a family member or friend for whom you are caring. Please care for him/her as you would care for someone in real life.

• In the room, this cabinet (point to cabinet) has your patient’s medical supplies that you may or may not need to deliver the dose of insulin. During this session, you are welcome to use all items available to you in the room at any time.

• Minimum items available to the participant in the cabinet:

• Insulin Pump User Manual

• An insulin pump

• Vial

• An infusion set

• A simulated sink

• Hand sanitizer

• Alcohol wipes

• A sharps container

• A trash can

• Clean towels

• Cotton balls and gauze

• Back-up vial

• Insertion device

• Eyewash kit

• Nickels, quarter (to remove battery)

• If there is anything else you need, please let me know, and I can either try to find it or make a note.

• During this session, we request that you perform all of the steps needed to deliver the medication dose, using the insulin pump. If there is something that you would normally do at home that you can’t do here, please let me know.

• Please bring a real-world mindset, while performing tasks.

• I may wait to answer your questions until the end of the session because I’m interested in exploring how you might resolve issues or interact with the product in real-life.

• Keep in mind that we are evaluating the device and not your individual skills or abilities.

• Do you have any questions before we begin?

2. Trained Users - Introductory Script

1. Introduction for the Baseline Assessment Session

• Hello, again, my name is ______________. I am the moderator, and I’ll be walking you through this session, and _______________ is the observer.

• First off, we want to thank you for your time and participation in this usability study.

• I’m going to start by asking a few informational questions:

• Based on the training you have received, how confident are you that you would be able to correctly deliver an insulin dose, using an insulin pump? Please give your rating on this scale from 1 - Not at All Confident to 5 - Very Confident.

• 1 - Not Confident, 2 - Unsure, 3 - Neutral, 4 - Confident, 5 - Very Confident

• Why did you rate it this way?

• I want you to imagine that you are a caregiver. The manikin, here, Sam (point to manikin) will represent your patient, which may be a family member or friend for whom you are caring. Please care for him/her as you would care for someone in real life.

• In the room, this cabinet (point to cabinet) has your patient’s medical supplies that you may or may not need to deliver the dose of insulin. During this session, you are welcome to use all items available to you in the room at any time.

• Minimum items available to the participant in the cabinet:

• Insulin Pump User Manual

• An insulin pump

• Vial

• An infusion set

• A simulated sink

• Hand sanitizer

• Alcohol wipes

• A sharps container

• A trash can

• Clean towels

• Cotton balls and gauze

• Back-up vial

• Insertion device

• Eyewash kit

• Nickels, quarter (to remove battery)

• If there is anything else you need, please let me know, and I can either try to find it or make a note.

• During this session, we request that you perform all of the steps needed to deliver the medication dose to Sam, using the insulin pump. If there is something that you would normally do at home that you can’t do here, please let me know.

• Please bring a real-world mindset, while performing tasks.

• I may wait to answer your questions until the end of the session because I’m interested in exploring how you might resolve issues or interact with the product in real-life.

• Keep in mind that we are evaluating the device and not your individual skills or abilities.

• Do you have any questions before we begin?

2. Introduction for the Second Evaluation Session

• Hello, my name is ______________. I am the moderator, and I’ll be walking you through this session, and _______________ is observing.

• The purpose of this session is to evaluate an insulin pump to identify whether you can use it safely in a simulated use environment.

• For purpose of this evaluation, we request that you turn off your phone. We appreciate if you do not use your phone throughout the session.

• If you feel unwell or wish to end the session at any point for any reason, please let me know.

• I’m going to start by asking a few informational questions:

• Based on the training you have received, how confident are you that you would be able to correctly deliver an insulin dose, using an insulin pump? Please give your rating on this scale from 1 - Not at All Confident to 5 - Very Confident.

• 1 - Not Confident, 2 - Unsure, 3 - Neutral, 4 - Confident, 5 - Very Confident

• Why did you rate it this way?

• For purpose of the session today, I want you to imagine that you are a caregiver. The manikin, here, Sam (point to manikin) will represent your patient, which may be a family member or friend for whom you are caring. Please care for him/her as you would care for someone in real life.

• In the room, this cabinet (point to cabinet) has your patient’s medical supplies that you may or may not need to deliver the dose of insulin. During this session, you are welcome to use all items available to you in the room at any time.

• Minimum items available to the participant in the cabinet:

• Insulin Pump User Manual

• An insulin pump

• Vial

• An infusion set

• A simulated sink

• Hand sanitizer

• Alcohol wipes

• A sharps container

• A trash can

• Clean towels

• Cotton balls and gauze

• Back-up vial

• Insertion device

• Eyewash kit

• Nickels, quarter (to remove battery)

• If there is anything else you need, please let me know, and I can either try to find it or make a note.

• During this session, we request that you perform all of the steps needed to deliver the medication dose, using the insulin pump. If there is something that you would normally do at home that you can’t do here, please let me know.

• Please bring a real-world mindset, while performing tasks.

• I may wait to answer your questions until the end of the session because I’m interested in exploring how you might resolve issues or interact with the product in real-life.

• Keep in mind that we are evaluating the device and not your individual skills or abilities.

• Do you have any questions before we begin?

3. Simulate Use Data (All participants)

Prompt:

• Please treat this as if you were actually administering a dose and follow ALL of the steps you normally would if you were administering a dose to someone at home.

• Just so you know, you will be injecting into this injection pad (indicate injection pad) that will be placed on Sam. I’ll set up the injection pad on Sam for you in the location that you select, whenever you are ready for that step.

• Now, I want you to imagine that you are the caregiver for Sam (point to manikin). Sam is a diabetic and has just been given this prescription from his/her doctor (show prescription card) and provided with an insulin pump set (point to pump and accessories in cabinet). Let’s imagine that you are at Sam’s home, and this is the first time you need to use the new pump to deliver Sam’s insulin dose.

• Sam’s doctors checked his/her blood-glucose levels and determined that the pump needs to be set up to deliver several doses of insulin throughout the day and should be set to deliver at 12am, 7am, and 1pm, as written on the prescription.

• Sam needs your help to set up the insulin pump to deliver the correct dose.

• When you are setting up the time and date on the pump, please set it to today’s date at 3:30 pm. For purpose of today’s session, we will imagine it is 3:30 pm when you set the time (show time prompt card).

• Please note that I may stop you at times to check something. This doesn’t mean you did something wrong; I just may not have been able to see what was going on.

• Whenever you are ready, please proceed to set up the insulin pump and deliver Sam’s first dose, based on the doctor’s prescription.

Study session start time: ________________

(If the participant reviews the User Manual before setting up any part of the pump)

Record duration of User Manual review: _________________

1. Basic Programming

1. Select either the 12-Hour set up or 24-Hour set up.

2. Set the time to 3:30 pm (12-Hour) or 15:30 (24-Hour).

Record Time set: ___________________

3. Set the date, month, and year to current date.

Record Date set: ___________________

2. Set up Reservoir and Vial

1. Remove the reservoir from package.

[Monitor for instances of participant’s touching reservoir septum]

2. Inspect vial to ensure liquid is free of particulates, colorless, and clear.

3. Check vial expiration date.

4. Swab the vial with alcohol.

5. Attach transfer guard to the vial with the plunger extended.

6. User does not compromise the sterility of the reservoir, vial, or transfer guard, during this step.

3. Filling the reservoir

1. Flip the vial over so that the vial is on top.

2. Pressurize the vial.

3. Slowly pull down on plunger rod and fill the reservoir to the number of units noted in the prescription card.

4. Gently tap the side of the reservoir to make any air bubbles rise to the top of the reservoir and slowly push up on the plunger to remove any air bubbles.

(if information was easy to record) Prompt: I am going to take a quick look and review the pump, since I wasn’t able to see everything that was going on.

Record reservoir fill level (mL): ___________________

5. Flip the vial over, upright, and disconnect the reservoir from the vial.

6. User avoids getting liquid on top reservoir or infusion set.

7. Dispose of transfer guard in sharps container.

8. Attach tubing connector onto the reservoir.

9. Twist to lock connector onto reservoir.

10. Push up on the plunger until you see insulin in the tubing to purge air bubbles from the reservoir.

11. Remove the plunger from the reservoir.

4. Pump Rewind & Insert into Pump case

1. Follow on-screen prompts to rewind the insulin pump plunger.

2. Insert the reservoir into the pump case unit.

3. Turn the connector clockwise, locking reservoir into the insulin pump.

5. Preparing the infusion set

1. Fill the infusion set tubing with the infusion set tubing disconnected from the body.

• If user does not see drops at the top of the needle after filling the infusion set tubing, user must select “No” on the pump screen and repeat filling until user sees drops.

(if needed) Prompt: Please let me know when you are ready to set up the infusion set tubing on the injection pad, and I will help you set up the injection pad.

6. Insert Infusion Set

1. Wash hands or use hand sanitizer.

Prompt: [when participant indicates that he or she is ready to use the injection pad] Where on Sam’s body would you do this injection? [set up the injection pad to reflect the choice on the manikin]

2. Clean the injection site on the injection pad with alcohol.

3. Secure the infusion set into the insertion device.

4. Remove the adhesive cover strips from infusion set.

5. Pull down on the green handle of the insertion device until it locks into place.

6. Safely remove the needle guard by twisting and pulling without sticking his or her finger.

7. Place the insertion device on the specified site on the patient.

8. Apply the infusion set to appropriate area on the manikin’s body by pressing the two blue buttons on the sides of the insertion device.

9. Press down on the grey button to release the infusion set from the insertion device.

10. Seal the infusion set by pressing adhesive onto skin.

7. Remove quick set needle

1. Pinch both sides of quick set and pull needle out.

2. Place the needle guard over needle.

3. Dispose of the infusion set needle in a sharps container.

8. Filling the Cannula

1. Fill the cannula until the pump beeps or vibrates, indicating cannula full.

9. Basal Dose (Program and deliver basal dose)

1. Set basal dose.

Record basal dose programmed: ___________________

2. Set basal rate 1.

Record basal 1 rate: ___________________

3. Set basal rate 2 and start time.

Record basal 2 time: ___________________

Record basal 2 time: ___________________

4. Set basal rate 3 and start time.

Record basal 3 time: ___________________

Record basal 3 time: ___________________

10. Bolus Dose (Program normal bolus dose)

Prompt: Now let’s imagine that Sam ate a very sugary snack and needed a large dose of insulin as soon as possible. The doctor noted the correct amount of insulin Sam needs for this type of situation on this prescription (give bolus prescription card). Please proceed to deliver the required insulin dose.

1. Navigate to the BOLUS MENU.

2. Select Normal Bolus.

3. Input desired bolus amount.

Record bolus dose: ___________________

4. Administer bolus dose.

11. Remove the Infusion Set

Prompt: Now, let’s imagine that Sam’s doctor just called and prescribed a different method for Sam to control his/her insulin. According to Sam’s doctor, he/she will not need to use the insulin pump for the next 2 weeks. Given that the insulin pump is currently attached to Sam, what would you do next?

1. Remove the used infusion set from the patient.

2. Remove the used reservoir from the infusion pump.

3. Dispose of the used infusion set in a sharps container.

4. Dispose of the used reservoir in a sharps container.

[Debrief if needed] What would you do with the [item(s)] after Sam was finished using the insulin pump for a few weeks?

12. Replace Battery

Prompt: Next, please imagine that the insulin pump battery is low and needs to be replaced. Please proceed to replace the existing battery with this new battery (provide participant with new battery).

1. Remove the battery cap.

2. Replace the battery with the new battery.

3. Fully tighten battery cap, so the slot is aligned horizontally with the pump.

13. (As time allows) Alarm Response

Prompt: For this next section, we’ll be simulating various troubleshooting scenarios. When you encounter these scenarios, please react as you normally would if you saw it, while you were treating Sam.

Prompt: Let’s imagine that you looked down at the insulin pump 30 seconds after you started a large insulin dose and saw this (show participant image of RESV alarm). What does this mean and what, if anything, would you do next?

1. Identify alarm: RESV alarm means the reservoir is low.

2. Identify corrective action: the reservoir needs to be replaced soon.

Prompt: Let’s imagine that you looked down at the insulin pump 30 seconds after you started a large insulin dose and saw this [show participant image of BOLUS STOPPED alarm]. What does this mean and what, if anything, would you do next?

3. Identify alarm: BOLUS STOPPED alarm means the dose delivery stopped.

4. Identify corrective action: the missed dose amount needs to be corrected for.

Prompt: Next, let’s imagine that you saw this on the insulin pump (show participant image of E alarm). What does this mean and what, if anything, would you do next?

5. Identify alarm: E alarm means there is an error with the pump.

6. Identify corrective action: contact a health care provider or the manufacturer for assistance.

Prompt: Next, let’s imagine that you saw this on the insulin pump half-way through an insulin injection (show participant image of EMPTY RESERVOIR alarm). What does this mean and what, if anything, would you do next?

7. Identify alarm: EMPTY RESERVOIR alarm means the reservoir is empty.

8. Identify corrective action: the reservoir needs to be changed and the missed dose corrected for.

[Preliminary Debrief on Use-Related Issues – DO NOT DEBRIEF DURING THE BASELINE ASSESSMENT]

4. Knowledge Tasks

Prompt: For this next section, there is some information that we want to make sure is communicated clearly. For these questions, we are not testing your memory, so you are welcome to do whatever you may typically do in real life to find the answers if you’re unsure.

1. Prompt: (optional) If you needed to change out the infusion tubing set, what would you do with the used components?

1. (as needed) What would you do with the reservoir? The infusion tubing set?

2. Prompt: What does a bolus dose do?

3. Prompt: What does a basal dose do?

[END SESSION HERE IF THIS IS THE PARTICIPANT’S BASELINE TEST. DO NOT DEBRIEF]

5. Subjective Feedback

1. Prompt: Did you experience any moments of hesitation, while using the product? [if yes, but participant doesn’t elaborate: What moments of hesitation did you experience?]

2. Prompt: What are some safety concerns you might have with this product, and how serious are they?

3. Prompt (if participant was trained): How much did you think about the training, during the study?

1. Please explain what parts of the training you thought about.

4. Prompt (if participant was trained): Was there anything that distracted you between the time you trained until the start of this session? [If yes, but participant doesn’t elaborate: What distracted you?]

5. Prompt (if participant was trained): Did you study any of the training materials or look up information on the product online after the training session? [If yes, but participant doesn’t elaborate: What did you do?]

6. Debrief on Use-Related Issues

[These questions should be used as guidance for debriefing on simulated use. Additional follow-up questions along the same lines may also be asked, based on specific observations or participant responses.]

• Did you happen to notice how the medication looked?

• [If yes] What did you look for?

• Did you happen to notice the expiration date on the product? What is the expiration date?

• Could you walk me through what happened?

• Do you have any concerns about what just happened? Is there a consequence, if any, that resulted from this situation?

• Is there something that could be changed to prevent this from happening?

• (if information was not recorded) I am going to take a quick minute to review the pump since I wasn’t able to see everything that was going on. (check pump time/date and bolus/basal history)

7. End of Session Script

1. Untrained Users - End of Session Script

Note: These prompts are intended for untrained participants, only, at the end of their evaluation.

Prompt: Before you leave today’s session, we have a short questionnaire for you to complete (hand participant laptop or tablet with end of session questionnaire). Please take a few minutes to complete the questionnaire and let me know if you have any questions.

(when participant has completed the end of session questionnaire)

Prompt: Thank you for completing the questionnaire. Now that the study is complete, I want to provide more insight on what we were looking for in the study. We are collecting data, so we can understand the impact of time between training and testing on task performance. Do you have any questions about this, general questions, or questions about the way the study was run?

Prompt: Please keep in mind that the simulation performed today does not represent actual device training, and you should not consider yourself better equipped to handle an insulin pump or other medical equipment, based on what you learned and saw in this simulation study. That’s it for our session today; thank you for your time.

2. Trained Users - Baseline Assessment - End of Baseline Session Script

Note: These prompts are intended for trained participants, only, at the end of their baseline assessment.

Prompt: That’s it for our session today; thank you for your time. Please keep in mind that the simulation performed today does not represent actual device training, and you should not consider yourself better equipped to handle an insulin pump or other medical equipment, based on what you learned and saw in this simulation study.

Prompt: I have noted that you are signed up to return for a second session (name date/time if available). You will receive your choice of the additional [incentive amount] or course credit if you are an SJSU student as compensation for returning and completing the second session. When you return for the next session, you may be asked to perform similar tasks. Since we’re interested in evaluating the use of a product at different intervals of time, please do not look up or otherwise seek out information on the product, during your break, as this may compromise our research results.

3. Trained Users - Main Study Evaluation - End of Session Script

Note: These prompts are intended for trained participants, only, at the end of their second evaluation session.

Prompt: Before you leave today’s session, we have a short questionnaire for you to complete (hand participant laptop or tablet with end of session questionnaire). Please take a few minutes to complete the questionnaire and let me know if you have any questions.

(when participant has completed the end of session questionnaire)

Prompt: Thank you for completing the questionnaire. Now that the study is complete, I want to provide more insight on what we were looking for in the study. We are collecting data, so we can understand the impact of time between training and testing on task performance. Do you have any questions about this, general questions, or questions about the way the study was run?

Prompt: That’s it for our session today; thank you for your time. Please keep in mind that the simulation performed today does not represent actual device training, and you should not consider yourself better equipped to handle an insulin pump or other medical equipment, based on what you learned and saw in this simulation study.

Note Taker Name: Page 1 of 25

Participant #: