2-3 EP attachment 6 - online discussion board evaluation protoc

OMB No. 0915-0212

Exp. Date 07/31/2021

A 90-Minute Online Bulletin Board Forum

EVALUATION PROTOCOL GUIDE

November 24, 2020

BACKGROUND AND INSTRUCTIONS

Thank you for your participation in this project! This project is sponsored by the Health Resources and Services Administration (HRSA). HRSA is an agency of the U.S. Department of Health and Human Services and is the primary federal agency for improving health care to people who are geographically isolated and economically or medically vulnerable.

The purpose of this Bulletin Board Forum is to discuss your thoughts and feelings about some health topics. We also want to hear your suggestions on how HRSA can best reach you with important health information. Your feedback will help develop important health information that is clearly written and easy to understand.

During the course of this 2-day discussion forum, please keep in mind the following:

• Your total time spent on this Bulletin Board Forum will be approximately 90 minutes (across 2 days).

• Day 1 will primarily ask you to share your thoughts on health messages and ask your preferences on how you like to receive health information. Day 1 questions will take you approximately 45 minutes to complete.

• Day 2 will ask you to share your thoughts on health education materials. Day 2 questions will take you approximately 45 minutes to complete.

• Six total participants will take part in this Bulletin Board Forum. You will be asked a set number of questions each day. After you answer each question, you will be able to see how others in the forum answered the same question. You will be able to react to comments from other participants in the Bulletin Board Forum as well as interact with the moderator. We want to hear all opinions and perspectives. Please be respectful of all opinions and perspectives.

• There are no right or wrong answers to the questions we ask.

• We ask you to type your responses to questions using a desktop or laptop computer or a tablet (this Bulletin Board Forum is not functional on a smartphone). At the end of the discussion, we will download everyone’s feedback for the purposes of our report. Please note that there will be NO audio or video recording of this Bulletin Board Forum.

• All information and opinions you give us will be kept strictly confidential. Your identity will not be revealed to anyone outside of this discussion group. Please do not share any information which could be used to identify someone, such as someone’s full name. This information will be used for research purposes only and will not be share with any third parties.

• We are not here to influence you or your behaviors in any way. What we seek is your honest feedback. You will not be targeted for any sales or promotional activities as a result of taking part in this discussion.

• Your participation is voluntary. What this means is that you are free to decline to answer any questions and you are free to withdraw from the Bulletin Board Forum at any time without penalty.

• Feel free to move through the questions at your leisure (at any time during the day). As your moderator, I may contact you on this Bulletin Board Forum to ask a follow-up question about something you wrote.

• Project staff may be viewing the Bulletin Board Forum, but the six participants and the moderator will be the only ones communicating during the 2-day discussion.

Thank you. Now, let’s get started!

DAY 1

INTRODUCTIONS

Please introduce yourself using your first name only (no last names) and let us know (a) what state you live in and (b) one thing you like to do for fun.

ATTITUDES AND EXPERIENCES

[PEOPLE WITH HIV] We are seeking to learn from you, as persons with HIV, about the health topics of our Bulletin Board Forum discussion.

[ADULTS 45+] We are seeking to learn from you, as adults 45 years of age and older, about the health topics of our Bulletin Board Forum discussion.

[CHILDREN AND FAMILIES] We are seeking to learn from you, as parents/guardians of at least one child between 6 months and 3½ years of age, about the health topics of our Bulletin Board Forum discussion.

1. [PEOPLE WITH HIV] How long ago were you diagnosed with HIV? PLEASE ANSWER USING YEARS AND MONTHS.

2. [ALL] Very briefly, what do the words “preventive health care” mean to you?

1. POLL: How important is preventive health care to you?

1-7 scale, 1 is “Not at all important” and 7 is “Extremely important.”

2. [PEOPLE WITH HIV] How does your rating response apply specifically to your HIV diagnosis? [OTHERS] Explain why you rated this question the way you did.

3. [ALL] What do the words “oral health” mean to you?

   1. What, if any, other parts of the body should be included in the broad “oral health” term? Why?

   2. Does “oral health” include any of the following: gums, throat, and bones around the mouth? Are any of these surprising? Why?

   3. What do you think people think of when they hear the term “oral health”? Do you think they know what is included within the term “oral health?” Is there a better way to talk about oral health that might make more sense to people?

4. POLL (PRE-TEST):

[PEOPLE WITH HIV and ADULTS 45+] How likely are you to visit the dentist in the next 6 months?

Why did you rate this question in this manner? Please talk about the extent to which your rating relates to the COVID-19/coronavirus pandemic.

[CHILDREN AND FAMILIES] How likely are you to take your child 6 months to 3½ years to visit the dentist in the next 6 months?

1 to 7 scale, 1 is “Not at all likely” and 7 is “Extremely likely.”

Why did you rate this question in this manner? Please talk about the extent to which your rating relates to the COVID-19/coronavirus pandemic.

5. [ALL] What, if any, recommendation have you received from a health care provider about how often to go [take your child] to the dentist for a check-up?

   1. Do you follow this recommendation? Why or why not?

   2. How often do you think you [very young children] should go to a dentist?

6. [ALL] In your own words, what is a “cavity”? What do you do when you have pain in your mouth, such as noticing you might have a cavity in any of your teeth? What determines how long you wait before you seek treatment?

7. POLL 3: How important is oral health to you?

1 to 7 scale, 1 is “Not at all important” and 7 is “Extremely important.”

1. [ALL] Explain why you rated this question the way you did.

2. [PEOPLE WITH HIV] How does this apply specifically to your HIV diagnosis?

3. [PEOPLE WITH HIV] How has your perception of oral health changed since you were first diagnosed with HIV?

4. [ALL] How do you think “good” or “bad” oral health can affect the rest of your body (beyond your mouth and teeth)? What other parts of your body can be affected by good or bad oral health?

HEALTH INFORMATION

[PEOPLE WITH HIV]

1. Please read the following sentence and then answer the questions below.

People living with HIV/AIDS are at increased risk for oral health problems.

1. In your own words, what is this sentence trying to say?

2. POLL. After reading this sentence, how likely are you to seek more information about this topic?

1 to 7 scale, 1 is “Not at all likely” and 7 is “Extremely likely.”

3. Please define, in your own words, what “increased risk” in the sentence means to you.

2. POLL. Which one of the following three statements is clearest to you? SELECT 1.

1. People living with HIV/AIDS are at increased risk for oral health problems.

2. People living with HIV/AIDS have a higher chance of experiencing oral health problems.

3. People living with HIV/AIDS are more likely to have oral health problems.

1. Please explain the reason for your response. How else would you write this statement so that it is even easier to understand?

3. Please read the following information and then answer the questions below.

Poor oral health may affect eating. This can lead to poor absorption of HIV medications.

1. In your own words, what is this information trying to say?

• In your own words, what does “poor absorption of HIV medications” mean?

2. POLL. After reading these sentences, how likely are you to seek more information about this topic?

1 to 7 scale, 1 is “Not at all likely” and 7 is “Extremely likely.”

3. POLL. How believable is this information?

1 to 7 scale, 1 is “Not at all believable” and 7 is “Extremely believable.”

4. What questions do you have after reading this information? What additional information would you want and why?

4. POLL. How likely are you to ask your dentist, in the next 6 months, about nutrition so that your medications can work as well as possible?

1 to 7 scale, 1 is “Not at all likely” and 7 is “Extremely likely.”

1. Explain why you rated this question the way you did.

[CHILDREN AND FAMILIES]

1. Please read the following sentence and then answer the questions below.

Healthy primary (baby) teeth help children chew food and speak clearly.

1. In your own words, what is this information trying to say?

2. What does “healthy” in this sentence mean to you?

2. Please read the following information and then answer the questions below.

Do not put your baby to bed with a bottle or sippy cup filled with breast milk, infant formula, fruit juice, or other liquids besides water. Liquids that contain sugar can cause tooth decay.

1. In your own words, what does this information mean to you?

2. What is an example of a “liquid that contains sugar”?

3. Do each of the liquids mentioned here (e.g., breast milk, infant formula, fruit juice) contain sugar? [Yes/No]

4. What surprises you about this information?

5. What additional questions do you have after reading this information?

3. Please read the following sentence and then answer the questions below.

Brush your child’s teeth after breakfast and before bed.

1. In your own words, what is this information trying to say?

2. How would you describe “after breakfast” / “before bed”—how would this work in your household?

3. How would you change this information to make it easier to understand?

4. Please read the following sentences and then answer the questions below.

After brushing, have your child spit out the remaining toothpaste but not rinse. The small amount of toothpaste that stays in the child’s mouth is good for the teeth.

1. In your own words, what is this information trying to say?

2. How much would you say a “small amount of toothpaste” is?

3. What does “…but not rinse” mean to you?

4. What surprises you about this information?

5. What additional questions do you have after reading this information?

[ADULTS 45+]

1. Please read the following information and then answer the questions below.

Oral health is important for people of all ages. A healthy mouth helps people enjoy their food, chew better, eat well, and avoid pain and tooth loss. With good oral hygiene and regular visits to the dentist, you can maintain your oral health for years.

1. In your own words, what does “hygiene” mean to you? And, what does “oral hygiene” mean to you?

   • Please provide any alternate terms you can think of for “oral hygiene”?

2. What does “regular” mean in this sentence? Please use numbers in providing your answer.

3. Take a look at the first sentence, “Oral health is important for people of all ages.” How believable is this information? Why?

4. Take a look at the last few words, “…you can maintain your oral health for years.”

   • What behaviors do you think fall under “maintaining” oral health? What activities do you need to do to maintain oral health?

5. When you read this information, did you think this was “speaking” to you? Why? In other words, who is the intended “audience” for this information?

2. Please read the following information and then answer the questions below.

Visiting the Dentist

• You or a person you care for should have regular dental appointments. Professional cleanings are just as important as brushing and flossing at home. Regular exams can identify problems early, before they cause unnecessary pain. Before each appointment, make sure to have a list of current medications, known allergies, and any insurance or billing information.

At these visits, the dentist or dental hygienist can suggest an oral hygiene routine that may make it easier for you or a person you care for. The dentist may also prescribe a special toothpaste or mouthwash to help prevent tooth decay, or mouthwash to fight germs that cause gum disease. Follow the dentist’s instructions for use.

1. What does “regular” in “regular dental appointments” mean to you? Please use numbers and dates in providing your answer.

2. What do “professional cleanings,” “oral hygiene routine,” “regular exams,” “germs,” and “gum disease” mean to you?

3. What do you think the sentence means by “special toothpaste”?

4. What is the role of “mouthwash” with respect to oral health? Does mouthwash help maintain oral health? Why or why not?

5. How often do you think this information would expect you to brush your teeth?

6. How often do you think this information would expect you to floss your teeth?

7. How realistic is it for you to follow the recommendations listed in this information? Why?

8. What other questions do you have about visiting the dentist after reading this information?

9. What changes could we make to this information to motivate you, more than you are now, to visit the dentist regularly?

COMMUNICATION

Next, we are interested in hearing about your preferences looking for oral health information.

Below is a list of sources people might use for oral health information. Please select the top three you trust the most to give you information about oral health and let us know why you picked these:

PROFESSIONAL ASSOCIATIONS/ORGANIZATIONS/INSTITUTIONS

• Association (Specify which one: ______)

• Hospital/health clinic

• Non-profit organization (Specify which one: ______)

HEALTH PROFESSIONALS/TRADITIONAL MEDICINE/SCIENCE EXPERTS

• Case manager

• Dental hygienist

• Dentist

• Doctor (primary care doctor or specialist doctor) (Specify which one: _____)

• Nurse

• Orthodontist

• Pharmacist

• Physician’s assistant

• Scientific journal (Specify which one: ______)

• Social worker

COMPLEMENTARY AND ALTERNATIVE THERAPIES

• Acupuncturist

• Chiropractor

• Herbalist/natural medicine

ACQUAINTANCES/LOVED ONES

• Co-worker/colleague

• Family member/spouse/partner

• Friend/peer

• Support group

PRINT MEDIA

• Books

• Magazine (Specify which one: ______)

• Newspaper (Specify which one: ______)

BROADCAST MEDIA

• Radio station (Specify which one: ______)

• Television station (Specify which one: ______)

INTERNET/SOCIAL MEDIA

• Google searches

• Online discussion group (Specify which one: ______)

• Social media (Specify which ones: Facebook, Instagram, Snapchat, Twitter, LinkedIn, etc.: _____)

• Websites (Specify which ones: ______)

• Website of medication manufacturer (Specify which ones: ______)

OTHER, please specify: ______: (1)__________________________________ , (2)__________________ and (3)________________________________.

Of the following, how do you prefer to receive oral health information? Please select your top two preferred formats and let us know why you picked them:

• Books

• Brochure

• Classroom setting (in-person or digital)

• Comic strip (comic with illustrations and line drawings, not photographs, and word balloons containing dialogue to tell a story)

• Email

• Infographic

• In-person conversation

• Fact sheet

• Fotonovela (comic with photographs and word balloons containing dialogue to tell a story)

• GIF

• Graphic novel (longer version of a comic strip or fotonovela)

• Podcast

• Poster

• Social media post (e.g., Facebook, Instagram, Snapchat, Twitter, LinkedIn)

• Regular mail

• Video

• Webinar

• Website

• Other, please specify: ______: (1)__________________________________ and (2)_____________________________________________________________.

If you only had the following two options, do you prefer reviewing health information in print form or in digital/electronic form? PICK ONLY ONE.

_____ Print form

_____ Digital/electronic form

Thank you for completing Day 1 of the Bulletin Board Forum. See you tomorrow for Day 2 of the Bulletin Board Forum!

DAY 2

Welcome back to the Bulletin Board Forum! Today is Day 2!

MATERIALS

[PEOPLE WITH HIV]

Please click here and take a few minutes to read this brochure: Oral Health: Tips for People Living with HIV. After you have reviewed this brochure, thank you in advance for responding to these prompts:

1. One thing in this brochure that grabbed your attention and why (for example, was something surprising or new to you?).

2. How clear and easy to understand is this brochure? If there was something confusing to you, let us know what it was, why, and any suggestions you might have for wording it more clearly.

3. POLL. To what degree do you feel like this brochure is speaking directly to you, like the brochure understands you and your situation?

1 to 7 scale, 1 is “Not at all relatable” and 7 is “Extremely relatable.”

4. How motivational this brochure is to you to change at least one behavior or action to maintain or improve your oral health (this includes learning more about this topic). What, if anything, will you change and when after reading this brochure?

Please pick one of the following two brochures to review:

1. Dry Mouth [CLICK HERE TO REVIEW]

2. Brushing and Flossing [CLICK HERE TO REVIEW]

After you have picked the brochure you would like to review, thank you in advance for responding to these prompts about the brochure THAT YOU HAVE SELECTED.

1. One thing in this brochure that grabbed your attention and why (for example, was something surprising or new to you?).

2. How clear and easy to understand is this brochure? If there was something confusing to you, let us know what it was, why, and any suggestions you might have for wording it more clearly.

3. POLL. To what degree do you feel like this brochure is speaking directly to you, like the brochure understands you and your situation?

1 to 7 scale, 1 is “Not at all relatable” and 7 is “Extremely relatable.”

4. How motivational this brochure is to you to change at least one behavior or action to maintain or improve your oral health (this includes learning more about this topic). What, if anything, will you change and when after reading this brochure?

[ADULTS 45+]

Please click here and take a few minutes to read this brochure: Oral Cancer. After you have reviewed this brochure, thank you in advance for responding to these prompts:

1. One thing in this brochure that grabbed your attention and why (for example, was something surprising or new to you?).

2. How clear and easy to understand is this brochure? If there was something confusing to you, let us know what it was, why, and any suggestions you might have for wording it more clearly.

3. POLL. To what degree do you feel like this brochure is speaking directly to you, like the brochure understands you and your situation?

1 to 7 scale, 1 is “Not at all relatable” and 7 is “Extremely relatable.”

4. How motivational this brochure is to you to change at least one behavior or action to maintain or improve your oral health (this includes learning more about this topic). What, if anything, will you change and when after reading this brochure?

Please pick one of the following two brochures to review:

1. Dry Mouth [CLICK HERE TO REVIEW]

2. Brushing and Flossing [CLICK HERE TO REVIEW]

After you have picked the brochure you would like to review, thank you for responding to these prompts about the brochure THAT YOU HAVE SELECTED.

1. One thing in this brochure that grabbed your attention and why (for example, was something surprising or new to you?).

2. How clear and easy to understand is this brochure? If there was something confusing to you, let us know what it was, why, and any suggestions you might have for wording it more clearly.

3. POLL. To what degree do you feel like this brochure is speaking directly to you, like the brochure understands you and your situation?

1 to 7 scale, 1 is “Not at all relatable” and 7 is “Extremely relatable.”

4. How motivational this brochure is to you to change at least one behavior or action to maintain or improve your oral health (this includes learning more about this topic). What, if anything, will you change and when after reading this brochure?

[CHILDREN AND FAMILIES]

Please pick one of the following two videos and one of the following two brochures to review:

VIDEOS (PICK ONE VIDEO TO REVIEW)

1. Brushing Toddlers’ Teeth video (approximately 2 minutes long) [CLICK HERE TO VIEW]

2. Lift The Lip to Prevent Decay video (approximately 2 minutes long) [CLICK HERE TO VIEW]

BROCHURES (PICK ONE BROCHURE TO REVIEW)

1. Taking Care of Your Baby’s Oral Health brochure [CLICK HERE TO REVIEW]

2. Brushing Your Child’s Teeth brochure [CLICK HERE TO REVIEW]

After you have picked one video and one brochure, respectively that you would like to review, thank you in advance for responding to these prompts FOR BOTH THE VIDEO AND THE BROCHURE THAT YOU HAVE SELECTED.

VIDEO

1. One thing in this video that grabbed your attention and why (for example, was something surprising or new to you?)?

2. How clear and easy to understand is this video? If there was something confusing to you, let us know what it was, why, and any suggestions you might have for wording it more clearly.

3. How motivational this video is to you to change at least one behavior or action to maintain or improve oral health (this includes learning more about this topic). What, if anything, will you change and when after viewing this video?

BROCHURE

1. One thing in this brochure that grabbed your attention and why (for example, was something surprising or new to you?).

2. How clear and easy to understand is this brochure? If there was something confusing to you, let us know what it was, why, and any suggestions you might have for wording it more clearly.

3. POLL. To what degree do you feel like this brochure is speaking directly to you, like the brochure understands you and your situation?

1 to 7 scale, 1 is “Not at all relatable” and 7 is “Extremely relatable.”

4. How motivational this brochure is to you to change at least one behavior or action to maintain or improve your oral health (this includes learning more about this topic). What, if anything, will you change and when after reading this brochure?

HEALTH INFORMATION

During these 2 days, you have reviewed at a lot of information.

Which ONE piece of information do you remember MOST clearly?

1. What was it about that information that made it stand out to you?

2. What do you remember about the wording/text/visuals/images of that information?

3. What do you feel the main idea of that information was trying to communicate?

4. POLL How likely are you to act on anything that was included in that information? And when?

Use a scale from 1 to 7, with 1 being “Not at all likely” and 7 being “Extremely likely.”

SOUND BITES

Please rank the following from most to least motivational (1=most motivational; 5=least motivational), meaning that the information inspires you to truly take an action. [If the itracks platform has this feature, we will randomize the presentation of these lists so that each participant sees the list in a different order/sequence.]

[ALL]

• Make an appointment with a dental provider to receive fluoride treatment.

• Make oral health care a regular part of taking care of yourself.

• Tell the dentist or hygienist if you notice any unusual changes in the way your mouth looks or feels or if it’s dry.

• Talk to your health care provider about developing an oral health plan.

• See your dentist regularly for check‐ups and cleanings.

Please rank the following from most to least motivational (1=most motivational; 5=least motivational), meaning that the information inspires you to truly take an action. [If the itracks platform has this feature, we will randomize the presentation of these lists so that each participant sees the list in a different order/sequence.]

[PEOPLE WITH HIV]

• Sip water or sugarless drinks often, especially during meals. This will make chewing and swallowing easier. It may also improve the taste of food.

• Avoid drinks with caffeine, such as coffee, tea, and some sodas (caffeine can dry out the mouth).

• Chew sugarless gum or suck on sugarless hard candy to stimulate saliva flow.

• Use a humidifier at night.

• Be aware that spicy or salty foods may cause pain in a dry mouth.

[CHILDREN AND FAMILIES]

• Begin brushing as soon as your child’s first tooth begins to show.

• _{Brush your child’s teeth after breakfast and before bed.}

• _{Clean your baby’s gums even before you can see the first tooth.}

• _{Use a child-sized toothbrush with soft bristles and fluoride toothpaste.}

• Gently brush your child’s teeth using small circles.

[ADULTS 45+]

• Gently brush at least twice a day with fluoride toothpaste.

• Floss regularly.

• Avoid sticky, sugary foods, or brush immediately after eating them.

• Angle the bristles toward the gumline so they clean between the gums and teeth.

• Brush gently back and forth in short circular strokes.

CLOSE

POLL (POST-TEST):

[PEOPLE WITH HIV and ADULTS 45+] How likely are you to visit the dentist in the next 6 months?

Why did you rate this question in this manner? Please talk about the extent to which your rating relates to the COVID-19/coronavirus pandemic.

[CHILDREN AND FAMILIES] How likely are you to take your child 6 months to 3½ years to visit the dentist in the next 6 months?

1 to 7 scale, 1 is “Not at all likely” and 7 is “Extremely likely.”

Why did you rate this question in this manner? Please talk about the extent to which your rating relates to the COVID-19/coronavirus pandemic.

Finally, please share any additional questions you have about oral health.

Those are all the questions we have. Thank you very much, again, for participating and be well!

Public Burden Statement: An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control number for this project is 0915-0212 and expires 07/31/2021. This Information Collections Request is voluntary. Public reporting burden for this collection of information is estimated to average 1.5 hours per response, including the time for reviewing instructions and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to HRSA Reports Clearance Officer, 5600 Fishers Lane, Room 14N136B, Rockville, Maryland, 20857.

20