2 Attach 4 CogInterview Screen & Pre-and Post-tests

Attachment #4:

Part 2: Cognitive Interview

Screening Script

Pre/Posttest Survey Questions with Probes

OMB No.: 0925-0642-04

Expiration Date: 9/30/2014

PRIVACY ACT NOTIFICATION STATEMENT

Collection of this information is authorized by The Public Health Service Act, Section 412 (42 USC 285 a-1). Rights of study participants are protected by The Privacy Act of 1974. Participation is voluntary, and there are no penalties for not participating or withdrawing from the study at any time. Refusal to participate will not affect your benefits in any way. The information collected in this study will be kept in private under the Privacy Act. Names and other identifiers will be separated from information provided and will not appear in any report of the study. Information provided will be combined for all study participants and report as summaries.

NOTIFICATION TO RESPONDENT OF ESTIMATED BURDEN

Public reporting burden for this collection of information is estimated to average 5 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to: NIH, Project Clearance Branch, 6705 Rockledge Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-0642). Do not return the completed form to this address.

Cognitive Interview

Screening Script

This subject population will consist of five individuals who are 18 or older, proficient in English, and non-Hispanic or Hispanic white

“Thank you for calling about the research study. The purpose of the Solar Cell study is to design a smart phone application, which uses smart phone technology to aid users in protecting their skin from damaging ultraviolet radiation (UV) in sunlight, a primary cause of skin cancer.

To see if you might qualify for this study, I need to ask you some questions. You do not have to answer any questions you do not want to answer. If you do not qualify for this study, the information you give me will be destroyed immediately.”

Screening Questions:

1. Are you 18 years of age or older?

   1. Yes (Continue)

   2. No (Thank you for your time)

2. How well do you read and speak English?

   1. Very well (Continue)

   2. Well (Continue)

   3. Not well (Thank you for your time)

   4. Not at all (Thank you for your time)

3. Are you:

   1. Hispanic or Latino

a. Not Hispanic or Latino

4. What is your race? (One or more categories may be selected)

a. White

b. Black or African American

c. American Indian or Alaska Native

d. Asian

e. Native Hawaiian or Other Pacific Islander

IF THEY DO NOT MEET ELIGIBILITY CRITERIA:

I’m sorry but we already have enough people with your characteristics for our cognitive interviews, so we do not need to meet with you. The information you provided to us will not be saved. Thank you for your interest in our research study. <Politely end the call>

If they meet eligibility criteria:

Great! You are eligible to participate in the study! We have several possible dates for the interview. Can you let me know which times work best for you from the following schedule?

Days Monday – Friday______________

Evenings Monday – Friday___________

Weekend Mornings________________

Weekend Afternoons_______________

Participant Contact Information:

First Name_________________________________________________________________

Home Phone________________________ Work Phone _________________________

Alternate Phone (Cell) __________________________

E-mail ________________________________________________________________

OMB No.: 0925-0642-04

Expiration Date: 9/30/2014

PRIVACY ACT NOTIFICATION STATEMENT

Collection of this information is authorized by The Public Health Service Act, Section 412 (42 USC 285 a-1). Rights of study participants are protected by The Privacy Act of 1974. Participation is voluntary, and there are no penalties for not participating or withdrawing from the study at any time. Refusal to participate will not affect your benefits in any way. The information collected in this study will be kept private under the Privacy Act. Names and other identifiers will be separated from information provided and will not appear in any report of the study. Information provided will be combined for all study participants and report as summaries.

NOTIFICATION TO RESPONDENT OF ESTIMATED BURDEN

Public reporting burden for this collection of information is estimated to average 60 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to: NIH, Project Clearance Branch, 6705 Rockledge Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-0642). Do not return the completed form to this address.

Cognitive Interview

Pre/Posttest Survey Questions with Probes

Probes for cognitive interviewing are in blue, however the prompts that are actually asked may change depending on the situation and the response from the participant.

1. Please estimate how many days in the past 3 months, that is from [date 3 months ago] until today, have you been outdoors in the sun between 10 am and 4 pm?

days

How sure are you of your answer?

How did you come up with that answer?

What does “been outdoors” mean to you?

2. About how many hours were your outdoors in the sun between 10 am and 4 pm in the past 3 months, that is from [date 3 months ago] until today?

hours per day

How difficult was it to answer this question? Why?

How sure are you of your answer?

How did you come up with that answer?

Of the [number days reported in Q1] days you said you were outdoors in the sun between 10 am and 4 pm in the past 3 months, on approximately how many of those days did you:

Don’t know/

Days remember

3. Wear sunscreen with SPF 15 or greater -99

4. Wear a sunscreen lip balm with SPF 15 or greater -99

5. Wear clothing that protected your skin from the sun -99

such as long sleeve shirts and long pants

6. Wear a hat with a wide brim -99

7. Wear sunglasses -99

8. Keep your time in the sun to a minimum -99

9. Stay in the shade -99

How did you come up with your answers?

How sure are you of your answers?

How easy was it to answer each item?

Would the question be easier to answer if we used a term like “approximately” at the beginning of the question statement?

Do you feel like providing an estimate would cause your answer to be more accurate, less accurate or about the same?

10. Has your skin been sunburned in the past 3 months between [date 3 months ago] and today? That is, has it been red and/or painful from exposure to the sun?

Yes -1

No -2

Don’t know/don’t remember -3

Did the clarifying statement make you think differently about what “sunburn” means? If so, how?

How sure are you of your answer?

How did you come up with that answer?

11. About how many times has your skin been sunburned in the past 3 months between [date 3 months ago] and today?

sunburns (record exactly)

Don’t know/don’t remember -999

How sure are you of your answer?

How did you come up with that answer?

Can you describe one or more instances of sunburn for me?

What did you consider “sunburned” as you answered this question?

Do you agree or disagree with the following statements?

	Strongly Agree	Agree	Neither Agree nor Disagree/ Don’t Know	Disagree	Strongly Disagree
12. Most of my family thinks getting a sun tan is a good thing. Who did you think about as you answered this question? Why did you pick the answer you did? What do you think of when you hear the phrase “sun tan”?	-5	-4	-3	-2	-1
13. Most of my family thinks people should protect their skin from the sun. How did you come up with your answer? Did you think of the same people or different people? What does “protect their skin from the sun” mean to you?	-5	-4	-3	-2	-1
14. Most of my friends think getting a sun tan is a good thing. Who did you think about as you answered this question? Why did you pick the answer you did?	-5	-4	-3	-2	-1
15. Most of my friends think people should protect their skin from the sun. Did you think about this question differently than question 13? If so, why? Did you think of the same people as question 14? How many people did you think of?	-5	-4	-3	-2	-1
16. I think I look healthier when I tan. Why did you pick the answer you did? What does the phrase “when I tan” mean to you?	-5	-4	-3	-2	-1
17. I think I look better when I tan. How did you come up with your answer? How was your answer to this question different than your answer to question 16? What I’m trying to get at is, what does being healthy and looking better mean to you? How are they different?	-5	-4	-3	-2	-1
18. It is so complicated to protect my skin from the sun that I still get sunburned. Why did you pick the answer you did? What did “complicated” mean to you in this question? If this question said “too complicated”, would that be better?	-5	-4	-3	-2	-1
19. Protecting my skin from the sun fits well with my outdoor activities. What did you think about in terms of “protecting my skin”? Why did you pick the answer you did? What activities did you think about? Are there any activities you didn’t think about or include when you answered?	-5	-4	-3	-2	-1
20. It is easy to protect my skin from the sun. Why did you pick the answer you did? What did “easy” mean to you in this question? What did “protect my skin from the sun” make you think of?	-5	-4	-3	-2	-1
21. I am confident that I can avoid getting sunburned while outdoors in the summer sun. Why did you pick the answer you did? What did you think about when you heard the term “outdoors”? How sure are you of your answer? Where does your confidence come from?	-5	-4	-3	-2	-1

22. How confident are you that you can practice sun safety, which is to wear sunscreen, protective clothing, a hat, and sunglasses, the next time you go out in the sun?

Very confident -4

Somewhat confident -3

A little confident -2

Not confident at all -1

Why did you pick the answer you did?

What does protective clothing mean to you?

What does the phrase “the next time you go out in the sun” mean to you?

Would you answer differently if we asked how confident are you that you will practice sun safety?

23. Over the next 3 months, how often do you think you will spend time in the sun in order to get a sun tan?

A lot -5

Often -4

Sometimes -3

Seldom -2

Never -1

Don’t know/remember -5

How sure are you of your answer?

What does “spend time in the sun in order to get a sun tan” mean to you?

How did you come up with your answer?

How easy is it to think about the next three months?

If we asked you at the beginning of the summer, how would you answer the question?

24. On the average, out of 100 people like you, how many do you think will:

Don’t know

a. Get sunburned while outdoors this summer -99

b. Intentionally try to get a sun tan this summer -99

c. Protect their skin from the sun this summer -99

Who did you think about when you saw the phrase “people like you”?

Did you think about different people or the same people as you answered a, b and c?

How sure are you of your answer?

How did you come up with your answer?

What does “intentionally try to get a sun tan” mean to you?
Did the order of the statements influence your answers?

25. Do you plan to protect your skin from the sun when you are outdoors in the future? That is, are you planning to apply sunscreen, wear protective clothing, wear a hat with a brim, limit your time out to the sun, and/or stay in the shade?

Yes -1

No -2

Don’t know -3

How did you come up with your answer?

What did “future” mean to you in this question?

Was there one type of protection that influenced your answer more than the others?

How sure are you of your answer?

Was this question easy or hard to answer?

Would this question be easier to answer if we asked about each type of sun protection individually?

26. How willing are you to protect your skin from the sun when you are outdoors in the future?

Very willing -1

Somewhat willing -2

Neither willing nor unwilling -3

Somewhat unwilling -4

Very unwilling -5

How did you come up with your answer?

What did “protect your skin from the sun” mean to you in this question?

What does “willing” mean to you in this question?

Was this question easy or hard to answer?

The next 3 questions will tell how sensitive your skin is to the sun.

27. What is your eye color?

Grey -1

Green -2

Blue -3

Hazel -4

Brown -5

Do you know why we asked this question?

28. What would happen to your skin if it was repeatedly exposed to bright sunlight in the summer without any protection? Would it:

Get no sun tan at all or only get freckled -1

Get mildly or occasionally tanned -2

Get moderately tanned -3

Get very brown or deeply tanned -4

How did you come up with your answer?

How sure are you of your answer?

What does the phrase “repeatedly exposed to bright sunlight in the summer” mean to you?

What did you think of when you saw the phrase “without any protection”?

How easy or difficult was this question to answer?
How would you rephrase it?

NOTE: Probe about what the answer choice means to the participant if they don’t describe it in the first probe.

29. What was your natural hair color before age 20, without artificial hair dye?

(If under 20, ask:) What is your natural hair color?

Red -1

Blonde -2

Light brown -3

Dark brown -4

Black -5

How did you come up with your answer?

How comfortable were you answering this question?

How honest would you be if you were asked this question out loud, in public?

Was this question easy or hard to answer?

What age did you picture yourself as of when you answered this question?

These last questions will be used simply to separate responses into groups.

30. What is your home zip code?

Zip code:

Refused -99999

33. How old are you?

years

Refused -999

How comfortable were you answering this question?

34. How many years of education have you completed?

11^th grade or less -1

High school graduate -2

Trade, technical, or vocational

Education beyond high school -3

Some college -4

2-year college degree (A.A.) -5

4-year college degree (B.A. or B.S.) -6

Post graduate degree -7

Refused -8

How comfortable were you answering this question?

How did you come up with your answer?

35. Has a doctor ever told you or anyone in your family that you or they had skin cancer, for example, basal cell or squamous cell cancer or melanoma?

Yes, basal cell or squamous cell cancer -1

Yes, melanoma -2

Yes, don’t know what type of skin cancer -3

No -4

Don’t know/remember -5

Refused -6

How comfortable were you answering this question?

How did you come up with your answer?

Who did you think of when you answered this question?

How sure are you of your answer?

What does “basal cell” mean to you? Squamous cell? Melanoma?

Would this question be easier to answer if we asked about you separately from your family?

36. Are you:

Male -1

Female -2

How comfortable were you answering this question?

37. Now speaking just of this past Saturday/Sunday, were you outdoors for longer than 15 minutes between 10 am and 3 pm? By outdoors, I mean not in a building and not in a covered vehicle.

(If intermittently outdoors, ask:)

Would you say you were actually outdoors for longer than 15 minutes in total?

(Randomly order asking about Saturday or Sunday)

Yes -1

No -2

Don't know/remember -3

How did you come up with your answer?

What time during the day were you thinking of?

What were you doing outside?

How did you remember what you were doing last weekend?

How sure are you of your answer?
Was this question easy or hard to answer?

(If “Yes” to Q37, continue with Q38; if “No” or “Don’t know/remember,” skip to Q57)

38. To the best of your recollection, how much of the sky was covered by clouds while you were outdoors last Saturday/Sunday?

0% (completely clear) -1

25% -2

50% -3

75% -4

100% (completely cloudy) -5

Don’t know/remember -6

How did you come up with your answer?

What time during the day or activity were you thinking of?

How sure are you of your answer?
Was this question easy or hard to answer?

39. What was the temperature when you were outdoors last Saturday/Sunday?

Cold -1

Cool -2

Warm -3

Hot -4

Very hot -5

Don’t know/remember -6

How did you come up with your answer?

What time during the day or activity were you thinking of?

How sure are you of your answer?
Was this question easy or hard to answer?

NOTE: Probe about what the answer choice means to the participant if they don’t describe it in the first probe.

40. Was your skin sunburned last Saturday/Sunday?

Yes -1

No -2

Don’t know/remember -3

How did you come up with your answer?

What does “sunburn” mean to you?

How did you remember if you were sunburned last weekend?

How sure are you of your answer?
Was this question easy or hard to answer?

41. Did you use a sunscreen with SPF 15 or greater on your skin last Saturday/Sunday?

Yes, used sunscreen -1

Yes, used makeup with sunscreen -2

No, didn't use sunscreen -3

Don't know/remember -4

How did you come up with your answer?

How did you remember if you wore sunscreen last weekend?

Do you always put sunscreen on the same way?

How sure are you of your answer?
Was this question easy or hard to answer?

(If “Yes” on Q41, continue with Q42; if “No” or “Don’t know/remember,” skip to Q44)

42. Did you apply the sunscreen before going out in the sun or after you'd been in the sun a while?

Before going out in the sun -1

After you'd been in the sun a while -2

Don't know/remember -3

How did you come up with your answer?

How did you remember when you put on your sunscreen last weekend?

What does “before going out in the sun” mean to you?

What does “after you’d been in the sun awhile” mean to you?

How sure are you of your answer?
Was this question easy or hard to answer?

43. Did you re-apply the sunscreen at any time on Saturday/Sunday?

Yes -1

No -2

Don’t know/remember -3

How did you come up with your answer?

How did you remember if you reapplied your sunscreen last weekend?

How sure are you of your answer?

44. Did you at any time choose to stay out of the sun so as not to get too much sun exposure last Saturday/Sunday?

Yes -1

No -2

Don’t know/remember -3

How did you come up with your answer?

How sure are you of your answer?
What does “too much sun exposure” mean to you?

What does “stay out of the sun” mean to you?

45. What activity were you doing most while outdoors last Saturday/Sunday between 10 am and 3 pm?

(record exactly)

Don’t know/remember -99

How did you come up with your answer?

How did you remember what activities you did last weekend?

How did you remember which activities you did between 10am and 3pm?

How sure are you of your answer?
Was this question easy or hard to answer?

46. Were you mostly in the shade or mostly out in the open and not in the shade while you were outdoors (Q45 activity)?

All in the shade/not in the open at all -5

Mostly in the shade/in the open some time -4

In the shade and in the open equally -3

Mostly in the open/in some shade -2

All in the open/not in shade at all -1

Don't know/remember -9

How did you come up with your answer?

How did you remember when you were in the shade during (Q20 Activity) last weekend?

What were you thinking about when you were deciding if you were in the shade or not?

How sure are you of your answer?

NOTE: Probe about what the answer choice means to the participant if they don’t describe it in the first probes.

47. Now, I'd like to know in detail what you were wearing when you were (Q37 activity) last Saturday/Sunday to find out how much of your skin was exposed to direct sunlight.

What clothing were you wearing on the upper or top part of your body?

Top/dress/shirt/blouse -1

Swimwear -2 (Go to Q49)

Nothing/topless -3 (Go to Q49)

Don't know/remember -4 (Go to Q49)

How did you come up with your answer?

How did you remember what you were wearing during (Q20 Activity) last weekend?

How sure are you of your answer?
NOTE: Probe about what the answer choice means to the participant if they don’t describe it in the first probes.

(If "top/dress/shirt/blouse" in Q47, ask:)

48. How long were the sleeves of that top/dress/shirt/blouse?

Long-sleeved -4

Three-quarter length -3

Elbow length -2

Short-sleeved -1

Sleeve-less -0

Don't know/remember -5

Can you show me by pointing at your arm how long your sleeves were?

How sure are you of your answer?
NOTE: Probe about what the answer choice means to the participant if they don’t describe it in the first probes.

49. What were you wearing on the lower part of your body when you were (Q37 activity) last Saturday/Sunday?

Trousers/jeans/shorts/skirt/dress -1

Swimwear/bathing suit -2 (Go to Q51)

Bottomless -3 (Go to Q51)

Don't know/remember -4 (Go to Q51)

How sure are you of your answer?

What does “bottomless” mean to you?
NOTE: Probe about what the answer choice means to the participant if they don’t describe it in the first probes.

(If "trousers/jeans/shorts/skirt/dress" in Q49, ask:)

50. How long were/was your trousers/jeans/shorts/skirt/dress?

Ankle length -3

Three-quarters length -2

Knee length -1

Above the knee -0

Don't know/remember -4

Can you show me by pointing at your leg how long your trousers/jeans/shorts/skirt/dress were?

How sure are you of your answer?
NOTE: Probe about what the answer choice means to the participant if they don’t describe it in the first probes.

(If "swimwear" in Q47 or Q49, ask Q51; if other clothing in Q47 and Q49 skip to Q52)

51. What sort of swimwear were you wearing?

One piece suit -6

Two-piece suit/bikini -5

Bikini top only -4

Bikini bottom only -3

Long shorts -2

Short shorts -1

Briefs/speedos -0

Don't know/remember -7

How sure are you of your answer?
NOTE: Probe about what the answer choice means to the participant if they don’t describe it in the first probes.

52. What kind of footwear, if any, were you wearing when you were (Q37 activity) last Saturday/Sunday?

Thongs/flip-flops -1

Sandals without socks/stockings -2

Sandals with socks -3

Sandals with stockings -4

Shoes without socks/stockings -5

Shoes with socks -6

Shoes with stockings -7

None -8

Don't know/remember -9

How sure are you of your answer?
How do you distinguish between thongs/flip-flops and sandals and shoes?

53. Were you wearing a cap, hat, or sun visor when you were (Q37 activity)?

Yes, hat -1

Helmet = hat with no brim

Yes, cap -2

Yes, visor -3 (Go to Q56)

No, none -4 (Go to Q56)

Don't know/remember -5 (Go to Q56)

How sure are you of your answer?
How do you distinguish between cap and hat and sun visor?

(If "hat" or "cap" in Q54, ask:)

54. Did your hat/cap have a brim or flaps on the front, sides, and/or back? (mark all that apply)

Front -1

Sides -2

Back -3

No brim/flaps -4 (Go to Q56)

Don't know/remember -5 (Go to Q56)

How sure are you of your answer?
Was this question easy to answer or hard to answer?

(If "brim" in Q54, ask:)

55. Was the brim of your hat/cap wide or narrow?

Yes, wide brim -1

A brim of 4" or more is considered "wide."

No, narrow brim -2

Don't know/remember -3

How sure are you of your answer?
How helpful is the “4 inches means wide” information?

56. Were you wearing sunglasses while you were (Q37 activity) last Saturday/Sunday?

Yes -1

No -2

Don't know/remember -3

Can you describe sunglasses to me?

How sure are you of your answer?

Finally, we want to ask you about the Solar Cell mobile application for smart phones that you were asked to download and use in this study.

57. Overall, I would rate the user-friendliness of the Solar Cell mobile application as:

Worst imaginable -0

Awful -1

Poor -2

OK -3

Good -4

Excellent -5

Best imaginable -6

Don’t know -7

I never used the Solar Cell

mobile application -8

How did you come up with your answer?

How sure are you of your answer?

Do you feel there are too many choices, too few choices or just enough?
NOTE: Probe about what the answer choice means to the participant if they don’t describe it in the first probes.

58. How helpful was the advice provided by Solar Cell mobile application for protecting your skin to avoid sunburn? Please rate the usefulness of each type of advice provided by Solar Cell.

Why does the phrase “helpful” mean to you in this question?

Why does the phrase “usefulness” mean to you in this question?

Does the lead of the question make sense with the answer choices?

	Very Helpful	Mostly Helpful	Somewhat Helpful	Only a Little Helpful	Not Helpful At All	I Never Saw That Advice
59. The timer that told you how much time it would take to sunburn your skin.	-5	-4	-3	-2	-1	-9
60. The information on how much vitamin D your skin was making from the sun.	-5	-4	-3	-2	-1	-9
61. The picture that showed how much sunscreen you needed to apply to your skin.	-5	-4	-3	-2	-1	-9
62. The reminder to re-apply your sunscreen.	-5	-4	-3	-2	-1	-9
63. The UV Index	-5	-4	-3	-2	-1	-9
64. The icon that indicated when UV A was high and you needed to protect yourself.	-5	-4	-3	-2	-1	-9
65. The planner that told you how quickly your skin would sunburn at a time in the next five days.	-5	-4	-3	-2	-1	-9
66. The warning to be careful when outdoors in the sun while taking medications that might make your skin more sensitive to the sun.	-5	-4	-3	-2	-1	-9

67. Will you to use the Solar Cell mobile application in the future to help you protect your skin and avoid sunburn?

Yes -1

No -2

Don’t know -3

How honestly do you think you would answer this question?

68. Would you recommend the Solar Cell mobile application to a friend or family member?

Yes -1

No -2

Don’t know -3

How honestly do you think you would answer this question?

69. Out of five stars, how would you rate the Solar Cell mobile application?

☼☼☼☼☼ -5

☼☼☼☼ -4

☼☼☼ -3

☼☼ -2

☼ -1

Don’t know -6

What would make you pick 5 stars versus 4 stars? 4 versus 3? 3 versus 2? 2 versus 1?

How honestly do you think you would answer this question?