Form Attachments C-F Attachments C-F Testing Protocol

Attachment C: CIPSEA Cognitive Interviewing Protocol

1. Introduction

1. Hi! Thank you for coming in today. My name is ( ). I work for the Office of Survey Methods Research in the Bureau of Labor Statistics. I look forward to working with you today.

2. With your permission, I will be taping this session. This allows me to ensure the accuracy of your answers, and also allows me to focus on you and your responses instead of taking notes. Do I have your permission to audiotape?

3. Before we get started, please read over the consent form. If you agree to participate in the study, print and sign your name as indicated on the bottom of the form. If you do not agree, please let me know now.

4. Let me explain a bit about what we’ll be doing today. The BLS collects a wide variety of data from companies and households to create important national statistics, such as the unemployment rate and the inflation rate. In order to earn, and keep, the trust of our respondents we work hard to keep all the information they give us safe and protected. We are actually required by law to protect the data.

5. Recent changes to the laws require us to make some changes in the way we explain how we keep data protected to our respondents. To ensure that the changes we make are clear and easy to understand to respondents, we are talking to you and many others to get your feedback to the wording.

6. We are looking for your honest feedback and reactions. There aren’t any right or wrong answers here, your opinions and thoughts are what matters today. We want to know what you would think if you heard the language before giving the BLS your personal information, so we will be asking you to pretend you’re a respondent who has been asked to participate in a BLS survey. With your ‘respondent hat’ on, I’ll be giving you the new language and then asking you a series of questions to get your feedback on it.

7. Do you have any questions before we begin?

1. Background

   1. Can you tell me a bit about what you do for a living?

   2. Have you answered government surveys before?

      1. For your company?

      2. For your household?

   3. If yes,

      1. Do you remember what surveys they were?

      2. What type of information did you provide?

      3. What do you remember about how the survey organization promised to keep your information secure?

   4. Did you have any concerns or hesitations about providing your information for the survey(s)?

2. Current CIPSEA Language: Verbal

1. Here is the way we currently explain how we protect data (read aloud)

The Bureau of Labor Statistics, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent.

2. I know that’s not the most straightforward thing you’ve heard, but can you tell me what it means in your own words?

3. What is your general reaction to this language?

4. If you heard this language before a survey, would it influence your decision to participate or not?

5. Do you have any concerns with how your (company’s / personal) information would be treated after hearing this?

3. Current CIPSEA Language: Visual

5. Here is the pledge I just read, I’ll give you a minute to read through it (show card)

The Bureau of Labor Statistics, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent.

6. Now that you’ve read it, can you again tell me what it means?

7. Is your reaction to it different now that you’ve seen it written down?

8. (take back show card) I want to ask you some questions about what the language covers and doesn’t cover. Please make your best guess, even if you’re not sure of the answer. If you gave BLS data and were assured confidentiality using this language, who would be able to see your information?

9. What would they be able to do with your information?

10. I’m going to read a list of groups, tell me if they’d be able to see the information you provided

    1. Bureau of Labor Statistics (BLS)

    2. Department of Labor (DOL)

    3. Internal Review Service (IRS)

    4. Social Security Administration (SSA)

    5. Department of Homeland Security (DHS)

    6. The Census Bureau

    7. All federal government agencies

    8. Congress

    9. Other

    10. Anyone else who would be able to access your data under this language?

11. Besides who can access your information, the language also talks about what can be done with it. Specifically it says it can only be used for “Statistical purposes.” What do you think that means?

12. I’m going to read a list of activities, tell me if you think they’d be allowed under this language?

    1. It could be combined with other survey answers to create summary statistics

    2. It could be published exactly as you provided it

    3. It could be given to other statistical agencies

    4. It could be given to the IRS

    5. It could be given to other federal government agencies

    6. It could be given to private companies

    7. What else could be done under this language?

4. Revised language {order will be randomly assigned across participants}

   1. As I mentioned, there have been changes to the laws related to the protection of BLS survey data. The main language that you just read will stay the same, but we’ll be adding the following to the end (show card)

To protect Federal information systems from cybersecurity risks, BLS information systems are protected through cybersecurity monitoring of the systems that transmit your data.

2. What does that mean to you?

3. What is your general reaction to it?

4. If you heard this language before a survey, would it influence your decision to participate or not?

5. Do you have any concerns with how your personal information would be treated after hearing this?

6. What does “cybersecurity risks” mean to you?

7. When it says that “BLS information systems” are protected, what does that mean?

   1. What would the monitoring be looking at?

   2. Would whoever is doing the monitoring be able to see the information you provided to the survey?

8. What about “cybersecurity monitoring”?

9. What do you think “the systems that transmit your data” are?

10. How do you think your data would be protected?

11. If you received this new language on a survey asking for information about your company would it impact your decision to respond?

    1. (Establishment version) What if the survey asked for

       1. Payroll information showing how much you pay your employees?

       2. Personal information about your employees?

       3. Proprietary information about your marketing plans?

       4. Is there any information you would be totally comfortable giving?

       5. Any information you’d be totally uncomfortable giving?

    2. (household version) What if the survey asked for

       1. Your name

       2. Your phone number

       3. Your income

       4. Your social security number

       5. Is there any information you would be totally comfortable giving?

       6. Any information you’d be totally uncomfortable giving?

12. Does this language change who could see the information you provided?

13. Any other comments?

5. Cybersecurity Risks vs. Malicious Activities

   1. We have a few other wording options we are considering. Instead of cybersecurity risks, we could say malicious activities. What would that mean to you?

   2. Does it change your overall reaction to the pledge? How so?

   3. Which do you recommend we use – cybersecurity risks or malicious activities?

6. BLS Information Systems vs. Your Data

   1. We also could say ‘your data’ instead of BLS information systems. {read language with your data} what do you think of that option?

   2. Does it change your overall reaction?

   3. Which one do you recommend we use?

7. Protected vs. Safeguarded

   1. We also could say ‘safeguarded’ instead of ‘protected’. {read language with your data} what do you think of that option?

   2. Does it change your overall reaction?

   3. Which one do you recommend we use?

8. Monitoring vs. Screening

   1. We also could say ‘screening’ instead of ‘monitoring’. {read language with your data} what do you think of that option?

   2. Does it change your overall reaction?

   3. Which one do you recommend we use?

9. Suggestions

   1. Data security is a top priority for the BLS, but we also don’t want to unnecessarily scare people. Do you have any suggestions on language that could help us achieve that balance?

   2. With all of these options, which do you think we should move forward with {show card}

10. Flier

    1. Finally, we have a flier that we’d give to respondents about this {hand flier}. What is your initial reaction?

    2. Does it clearly communicate the things we’ve talked about so far?

    3. Does anything stand out to you about the content?

    4. Do you think it would be useful?

    5. Is there anything we should add? Remove?

11. Conclusions

    1. We’re almost done, thanks for your feedback so far. In general, do you have any concerns about how the government treats the data you provide for surveys?

    2. Do you have any suggestions or other feedback ?

Attachment D: TryMyUI Web Survey

Page 1

Introduction

Unlike some tasks you’ve completed on TryMyUI, we’re not looking for you to evaluate a website, rather we’re looking for you to answer a series of questions and give us your written and verbal feedback. Please talk aloud, giving your reactions and thoughts as you move through the pages.

This is a survey for the US Bureau of Labor Statistics. This voluntary study is being collected by the Bureau of Labor Statistics under OMB No. 1220-0141. This survey will take approximately 20 minutes to complete. Your participation is voluntary and you have the right to stop at any time.

We are looking for information about how respondents answer our surveys. Please take your time as you answer these questions. The information you provide will contribute to valuable research at the BLS, one of the leading federal statistical agencies.

{continue button}

This survey is being administered by Qualtrics and resides on a server outside of the BLS Domain. The BLS cannot guarantee the protection of survey responses and advises against the inclusion of sensitive personal information in any response.

Page 2

The Bureau of Labor Statistics of the U.S. Department of Labor is the principal Federal agency responsible for measuring labor market activity, working conditions, and price changes in the economy.

Its mission is to collect, analyze, and disseminate essential economic information to support public and private decision-making.

As an independent statistical agency, BLS serves its diverse user communities by providing products and services that are objective, timely, accurate, and relevant.

1. Prior to today, had you heard of the Bureau of Labor Statistics?

   1. Yes

   2. No

2. Have you ever completed a survey for a market research firm, polling company or other private company?

   1. Yes

   2. No

3. Have you ever completed a survey for the US Government?

   1. Yes (go to 4)

   2. No (go to 5)

4. Did you complete the US Government survey online, in person or on the phone?

   1. Online

   2. In-person

   3. Over the phone

5. In the last year, how many surveys have you completed?

   1. Fewer than 5

   2. 5 to 10

   3. 11 to 20

   4. More than 20

6. Prior to today, how much have you thought about survey companies protect the information you provide?

   1. A lot of thought

   2. Some thought

   3. Never thought about it

7. Please read the below information about confidentiality at the BLS:

Confidentiality at the BLS: {Insert one of pledge versions, randomly assigned}

Please click to indicate that you’ve read the above information about confidentiality at the BLS.

{I have read about confidentiality at the BLS}

1. In your own words, please summarize the information about confidentiality at the BLS.

2. What is your general reaction to this language?

3. Would the information about confidentiality at the BLS play a role in your decision to participate in a BLS survey or not?

   1. Yes

   2. No

4. Thinking only about confidentiality, how likely are you to participate in a BLS survey in the future now that you’ve read the information about confidentiality at the BLS?

   1. Very likely

   2. Somewhat likely

   3. Not at all likely

5. Do you have any concerns about how BLS would keep your information confidential?

   1. Yes

   2. No

6. Under this pledge, who would have access to the information you provided on a BLS survey? (mark all that apply)

   1. Bureau of Labor Statistics (BLS)

   2. Department of Labor (DOL)

   3. Internal Review Service (IRS)

   4. Social Security Administration (SSA)

   5. Department of Homeland Security (DHS)

   6. The Census Bureau

   7. All government agencies

   8. Congress

   9. Other

7. Under this pledge, what could be done with the information you provided on a BLS survey? (mark all that apply)

   1. It could be combined with other survey answers to create summary statistics

   2. It could be published exactly as you provided it

   3. It could be given to other statistical agencies

   4. It could be given to the IRS

   5. It could be given to other federal agencies

   6. It could be given to private companies

Page 3

Next, we’d like to give you more background information for the following task.

The BLS collects a wide variety of data from companies and households to create important national statistics, such as the unemployment rate and the inflation rate. In order to earn, and keep, the trust of our respondents we work hard to keep all the information they give us safe and protected. We are actually required by law to protect the data.

Recent changes to the laws require us to make some changes in the way we explain how we keep data protected to our respondents. To ensure that the changes we make are clear and easy to understand to respondents, we’d like to get your feedback on wording options for our confidentiality statement.

We have a few wording options. You’ll see a pair of words, please discuss your thoughts about each and which one you’d recommend using and why.

To protect Federal information systems from cybersecurity risks [malicious activities], BLS information systems [your data] are protected [safeguarded] through cybersecurity monitoring [screening] of the systems that transmit your data.

8. Cybersecurity risks vs. Malicious activities

9. BLS information systems vs. Your data

10. Protected vs. Safeguarded

11. Monitoring vs. Screening

Page 4

12. Any final comments or thoughts about confidentiality at the BLS?

Attachment E: MTurk Web Survey

MTurk CIPSEA Survey

5.27.16

Page 1

Introduction

This is a survey for the US Bureau of Labor Statistics. This voluntary study is being collected by the Bureau of Labor Statistics under OMB No. 1220-0141. This survey will take approximately 5 to 10 minutes to complete. Your participation is voluntary and you have the right to stop at any time.

We are looking for information about how respondents answer our surveys. Please take your time as you answer these questions. The information you provide will contribute to valuable research at the BLS, one of the leading Federal statistical agencies.

This survey is being administered by Qualtrics and resides on a server outside of the BLS Domain. The BLS cannot guarantee the protection of survey responses and advises against the inclusion of sensitive personal information in any response. By proceeding with this study, you give your consent to participate in this study.

{continue button}

Page 2

The Bureau of Labor Statistics of the U.S. Department of Labor is the principal Federal agency responsible for measuring labor market activity, working conditions, and price changes in the economy.

Its mission is to collect, analyze, and distribute essential economic information to support public and private decision-making.

As an independent statistical agency, BLS serves its diverse user communities by providing products and services that are objective, timely, accurate, and relevant.

8. Prior to today, had you heard of the Bureau of Labor Statistics?

   1. Yes

   2. No

9. Have you ever completed a survey for a market research firm, polling company or other private company?

   1. Yes

   2. No

10. Have you ever completed a survey for the US Government?

    1. Yes (go to 4)

    2. No (go to 5)

11. Did you complete the US Government survey online, in person or on the phone?

    1. Online

    2. In-person

    3. Over the phone

12. In the last year, outside of MTurk, how many surveys have you completed?

    1. Fewer than 5

    2. 5 to 10

    3. 11 to 20

    4. More than 20

13. Prior to today, how much have you thought about how survey companies (the government or otherwise) protect the information you provide?

    1. A lot of thought

    2. Some thought

    3. Never thought about it

14. Without looking for any information, please tell us how do you think the government keeps survey data confidential?{open ended}

Page 3

15. Please read the below information about confidentiality at the BLS:

Confidentiality at the BLS: {Insert one of pledge versions, randomly assigned}

Please click to indicate that you’ve read the above information about confidentiality at the BLS.

{I have read about confidentiality at the BLS}

Page 4

Now, we’ll ask you a few questions about yourself.

13. What is your age?

14. What is the highest level of education you’ve completed?

    1. Less than high school

    2. High school diploma or GED

    3. Some college

    4. Associate degree

    5. Bachelor’s degree

    6. Graduate school degree

    7. Prefer not to say

15. Are you currently employed:

    1. Full time (go to 4)

    2. Part time (go to 4)

    3. Unemployed (go to 6)

    4. Student (go to 6)

    5. Retired (go to 6)

    6. Other, specify (go to 4)

    7. Prefer not to say

16. What is your occupation?

    1. Administrator, manager

    2. Teacher

    3. Professional

    4. Administrative support, including clerical

    5. Sales

    6. Technician

    7. Protective Services

    8. Other service

    9. Machine or transportation operator

    10. Construction worker, mechanic

    11. Farming

    12. Forestry, fishing or grounds keeping

    13. Armed Forces

    14. Other, specify:______

    15. Prefer not to say

17. What is your job title?

    1. Prefer not to say

18. What is your annual income?

19. If 6 is blank, In which of these ranges does your annual income fall?

    1. Less than $25,000

    2. $25,000 - $50,000

    3. $50,001 – $75,000

    4. $75,001 - $100,000

    5. More than $100,000

    6. Prefer not to say

20. What is your annual household income?

21. If 6 is blank, In which of these ranges does your annual household income fall?

    1. Less than $25,000

    2. $25,000 - $50,000

    3. $50,001 – $75,000

    4. $75,001 - $100,000

    5. More than $100,000

    6. Prefer not to say

    7. Don’t know

22. How many people currently live in your home?

    1. Prefer not to say

23. Are you of Hispanic, Latino or Spanish origin?

    1. Yes

    2. No

    3. Prefer not to say

24. What is your race?

    1. White

    2. Black or African American

    3. American Indian or Alaska Native

    4. Asian

    5. Other

    6. Prefer not to say

25. What is your zip code?

    1. Prefer not to say

Page 5

26. Earlier in this survey, you were shown information about confidentiality at the BLS. Do you remember seeing that?

    1. Yes

    2. No (skip to top of page 7)

27. In your own words, please summarize the information about confidentiality at the BLS.

28. Please list as many words or phrases as you can from that information:

Page 6

29. Please check the words that were part of the information about confidentiality:

    1. Bureau of Labor Statistics (BLS)

    2. Department of Labor (DOL)

    3. Internal Revenue Service (IRS)

    4. Department of Homeland Security (DHS)

    5. Cybersecurity

    6. Threat

    7. Risk

    8. Terrorist

    9. Mandatory

    10. Voluntary

    11. Law

    12. Statistical

    13. Security

    14. Consent

    15. Confidential

Page 7

Here is the text about how your information is kept confidential at the BLS again {show one of the alternate versions}.

30. Now that you see the text again, please rephrase it using your own words.

31. Do you have any concerns about how BLS would keep your information confidential?

    1. Yes

    2. No

32. Would how the BLS keeps your information confidential play a role in your decision to participate in a BLS survey or not?

    1. Yes

    2. No

33. According to this pledge, who would have access to the information you provided on a BLS survey? (mark all that apply)

    1. Bureau of Labor Statistics (BLS)

    2. Department of Labor (DOL)

    3. Internal Revenue Service (IRS)

    4. Social Security Administration (SSA)

    5. Department of Homeland Security (DHS)

    6. The Census Bureau

    7. Congress

    8. The White House

    9. All government agencies

    10. Other, please specify:

34. According to this pledge, what could be done with the information you provided on a BLS survey? (mark all that apply)

    1. It could be combined with other respondents’ survey answers to create statistics

    2. It could be given to other statistical agencies for policy making

    3. It could be given to other federal agencies for law enforcement

    4. It could be given to the IRS for tax enforcement

    5. It could be given to private companies for telemarketing

    6. Other

Page 8

Next, we’d like to give you more background information for the following task.

The BLS collects a wide variety of data from companies and households to create important national statistics, such as the unemployment rate and the inflation rate. In order to earn, and keep, the trust of our respondents we work hard to keep all the information they give us safe and protected. We are actually required by law to protect the data.

Recent changes to the laws require us to make some changes in the way we explain how we keep data protected to our respondents. To ensure that the changes we make are clear and easy to understand to respondents, we’d like to get your feedback

We have a few wording options. You’ll see a pair of words, mark which one you’d recommend using, and then answer a few questions about each.

To protect Federal information systems from cybersecurity risks [malicious activities], BLS information systems [your data] are protected [safeguarded] through cybersecurity monitoring [screening] of the systems that transmit your data.

35. Cybersecurity risks vs. Malicious activities

	Cybersecurity risks	Malicious activities
Which should we use?
Which is easier to understand?
Which is a more common problem for government information systems?
Which sounds least threatening?

35a. Comments

36. BLS information systems vs. Your data

	BLS information systems	Your Data
Which should we use?
Which is easier to understand?
Which sounds least threatening?

36a. Comments

37. Protected vs. Safeguarded

	Protected	Safeguarded
Which should we use?
Which is easier to understand?
Which sounds least threatening?

37a. Comments

38. Monitoring vs. Screening

	Monitoring	Screening
Which should we use?
Which is easier to understand?
Which sounds least threatening?

38a. Comments

Page 11

1. Any final comments or thoughts about confidentiality at the BLS?

Attachment F: Data Collection Center Web Survey

CIPSEA Phase 1 Research Goals:

• To understand how the CIPSEA pledge is currently administered

• To understand how respondents currently react to the pledge as administered

• To get feedback on the proposed alternative revisions from both FEs and proxy-respondents

Survey Goals

• To determine what, if any, experience the DCC staff has with interacting with respondents about the CIPSEA pledge and confidentiality concerns

Sample

• 20 full time DCC staff in Kansas City

• 20 full time DCC staff in Dallas

Timeline

• Distribute after OMB approval (likely early July)

• Close out 1 week later

Introduction

As you probably know, the BLS collects its data under the Confidential Information Protection and Statistical Efficiency Act of 2002, or CIPSEA. To accommodate recent legislation, we are testing language that may be added to the CIPSEA pledge. Before any changes are made, it is important to understand how the pledge is currently administered and what reactions, if any, respondents have to it. To determine this, we are reaching out to a variety of BLS staff to identify who has direct experience with respondent reactions to the confidentiality pledge.

Do not include any personally identifiable information (PII). This survey is being administered by surveymonkey.com and resides on a server outside of the BLS domain. BLS cannot guarantee the protection of survey responses and advises against the inclusion of sensitive personal information in any response.

Please complete this survey by XXX.

Background

1. What is your role at the Data Collection Center?

   1. Interviewer 1

   2. Interviewer 2

   3. Supervisor

   4. Task Manager

   5. Other, specify

2. How often do you directly interact with respondents?

   1. Daily (Go to 3)

   2. Weekly (Go to 3)

   3. Monthly (Go to 3)

   4. Only a few times a year (Go to 7?)

3. When you interact with respondents, what are you most often doing?

   1. Answering questions (Go to 4)

   2. Collecting data (Go to 5)

   3. Gaining cooperation (Go to 5)

   4. Other, specify (Go to 5)

4. What are the topics of questions that respondents ask you? (open ended)

Confidentiality Concerns

5. Have any respondents expressed concerns to you about how their data is protected?

   1. Yes (Go to 6)

   2. No (Go to 7)

6. How do you usually respond? (open ended)

CIPSEA

7. Are you familiar with the current CIPSEA pledge?

   1. Yes

   2. No (go to 10)

8. Do you ever talk to respondents about the CIPSEA pledge?

   1. Yes

   2. No (go to 10)

9. In what context do you talk about CIPSEA with respondents (open-ended)

10. Do you have any comments or questions related to confidentiality and the CIPSEA pledge? (Open ended)