2017 Integrated Postsecondary Education Data System (IPEDS) Time Use and Burden Cognitive Interviews Round 2

National Center for Education Statistics

Attachments

2017 Integrated Postsecondary Education Data System (IPEDS)

Time Use and Burden Cognitive Interviews Round 2

OMB# 1850-0803 v.192

March 2017

Attachments:

Attachment I – Respondent Recruitment Procedures and Materials

Attachment II – Consent to Participate in Research

Attachment III – Cognitive Interview Protocol – Round 2

Attachment IV Handout of Test Questions

Attachment V – Cognitive Interview Survey Items

Attachment I: Respondent Recruitment Procedures and Materials

Recruitment Procedures:

Participants will be recruited from institutions that completed IPEDS. Selected participants will be those institution staff who are responsible for collecting and providing the requested data for IPEDS. Sampled individuals will be contacted initially by email and/or telephone by AIR project staff. They will be invited to participate in an interview session, which will take place via video conference, conference call, or in-person at the institution. Each session will last a maximum of 60 minutes.

Participants for interviews will be recruited from institutions throughout the United States and near American Institutes for Research (AIR) Washington D.C. headquarters. To the extent possible, participants will be selected so that a range of institution sizes and institutional levels are represented.

Recruitment Materials:

Recruitment by Email:

Hello [Addressee],

I am writing to you from the American Institutes for Research (AIR) on behalf of the National Center for Education Statistics (NCES). NCES would like to interview [Institution] staff in order revise the time and burden questions at the end of each IPEDS survey to more accurately collect this information in the future.

During the interview, staff from the NCES contractor, AIR, will ask you questions on how you prepare for the IPEDS collections, your processes, and how you report time and burden estimates. We expect it will take about 60 minutes to complete the interview and your participation is voluntary.

If you are interested please email [email protected] or call ###-###-#### to set up an interview time.

Thank you,
AIR IPEDS Research Team

Recruitment by Phone:

Hello,

I am calling from the American Institutes for Research on behalf of the National Center for Education Statistics, or NCES, part of the U.S. Department of Education. NCES would like to interview institution staff in order revise the time and burden questions at the end of each IPEDS survey to more accurately collect this information in the future.

During the interview, staff from the NCES contractor, AIR, will ask you questions on how you prepare for the IPEDS collections, your processes, and how you report time and burden estimates. We expect it will take about 60 minutes to complete the interview and your participation is voluntary.

Do you have any questions?

[Pause]

Would you be willing to participate in this interview?

[If yes] Fantastic. Thank you for agreeing. Your input will help ensure an improved time and burden reporting for the IPEDS surveys. As the next step, we would like to set up a date and time for your interview. When are you available?

[Record date and time]

Thank you for your participation, we will send you a confirmation email.

[If no] We know that there are many demands on your time. I completely understand, thank you for your time.

Confirmation Email:

Good morning/afternoon [Respondent],

Thank you for your commitment to participate in an interview about IPEDS time use and burden.

 We look forward to seeing you on [date] at [time].

 The session will last approximately 60 minutes.

• The session will take [place in-person/video conference/conference call] at [address/location – call in number/video conference website].

• The purpose of the interview is to improve the accuracy of time use and burden estimates provided by respondents on IPEDS surveys.

Your participation in this study is completely voluntary, but we greatly appreciate your willingness to help improve our IPEDS data collection systems. If you have any questions, please contact us at [number] or [email].

Thank you,

[Name]

Attachment II: Consent to Participate in Research

Integrated Postsecondary Education Data System (IPEDS): Time Use and Burden Study

Introduction and Purpose

You, along with participants from other institutions, are being asked to participate in a cognitive interview being carried out by the American Institutes for Research (AIR) for the National Center for Education Statistics (NCES), part of the U.S. Department of Education. The purpose of the study is to interview institution staff experienced with the collection of IPEDS to explore reporting time use and burden. The results of these interview sessions will be used to improve questions collecting time use and burden responses in future IPEDS surveys.

Procedures

You are one of approximately 24 institutions who will be taking part in this study. Participants from other institutions will be asked similar questions.

The interview session will be recorded to make sure we don’t miss anything that you say and to help us write a report summarizing the results of the discussions. Upon completion of the written report, the recording will be destroyed. Your name, and the name of your institution, will never be used in the written report.

Study Duration

Your participation in the cognitive interview will take about 60 minutes.

Possible Risks or Discomforts

We do not anticipate that any of the discussion topics will make you uncomfortable or upset. However you may refuse to answer any question or take a break at any time.

Confidentiality

AIR is carrying out this study for the National Center for Education Statistics (NCES) of the U.S. Department of Education. NCES is authorized to conduct this study by the Education Sciences Reform Act (20 U.S.C., § 9543). Your participation is voluntary. All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

Your Rights

Your decision to take part in this research study is completely voluntary. You can refuse any part of the study and you can stop participating at any time.

Your Questions

If you have any questions about the study, you may call [Insert AIR employee name] at ###-###-####. If you have any questions about your rights as a study participant, you may call the Chair of AIR’s Institutional Review Board at 1-800-634-0797.

YOU WILL BE GIVEN A COPY OF THIS CONSENT FORM TO KEEP.

Your signature below indicates that you have read the information provided above, have received answers to your questions, and have freely decided to participate in this research. By agreeing to participate in this research, you are not giving up any of your legal rights.

________________ Date _____________________________________ Signature of Participant

_____________________________________

Printed Name of Participant

I certify that the nature and purpose, the potential benefits, and possible risks associated with participating in this research have been explained to the above-named individual.

________________ Date _____________________________________ Signature of Person Obtaining Consent

_____________________________________

Printed Name of Person Obtaining Consent

Attachment III: Cognitive Interview Protocol – Round 2

Name(s)/Title(s):

Institution:

Email:

Phone:

Thank you for taking the time to speak with us today. My name is__________, and my colleague, ___________, are members of the American Institutes for Research (AIR) project team conducting interviews to improve upon the question collecting time and burden reports in Integrated Postsecondary Education Data System (IPEDS). AIR, a research firm headquartered in Washington, D.C., has been contracted by the National Center for Education Statistics (NCES) to learn about your experiences reporting time and burden for IPEDS.

NCES would like to better collect and record time and burden from individuals who report data for IPEDS. You were chosen to participate in this second round interview because we value your opinion and would like to hear your perspective about time and burden reporting for IPEDS. We recognize that everyone has different experiences with IPEDS and want to stress that there are no right or wrong answers. In order for us to advise NCES on how to improve the collection of time and burden for IPEDS, we need to hear your thoughtful and honest feedback.

The interview should take about 60 minutes. I will be asking the questions, and _______________ will be taking notes. We would like to record our conversation to make sure that we catch all of the important information that you will share with us. The recording will serves as our back up to the notes that _______________ will take today. NCES will have access to the recording. Is it okay for me to record you?

[Once recording starts] Now that we are recording, could you answer these 2 questions: Do you consent to be part of this study? Is it okay for us to record our session today?

Do you have any questions before we get started?

Background Information:

• Before we begin, we’d like to get some background information about your role(s) at [institution name].

• How would you describe your overall role(s) in the institution?

• What are your day-to-day activities?

• How would you describe your responsibilities for IPEDS?

• How were you selected to report your institution’s data through IPEDS?

• Did you complete any special training or certification for IPEDS data collection and entry?

• How long have you been involved with IPEDS?

Currently Reporting Time and Burden:

• Currently there is one question regarding burden at the end of each of the 11 IPEDS surveys.

Example:

• In your own words, what is this question asking you?

• How did you arrive at your answer?

• How certain are you of your response?

• What activities come to mind when you answer this question?

• What does “prepare” mean to you?

• If you were to replace “prepare” in this question, what word would you use instead?

• How would you interpret this question if “institution” was added? Revised statement would read, “How long did it take your institution to prepare this survey component?”

• Additionally, there are currently two sentences below the time and burden question that describe why IPEDS collects the amount of time it takes to prepare the survey:

“The time it took to prepare this component is being collected so that we can continue to improve our estimate of the reporting burden associated with IPEDS. Please include in your estimate the time it took for you to review instructions, query and search data sources, complete and review the component, and submit the data through the Data Collection System.”

• Do you recall reading these sentences when you last completed IPEDS?

• Would you find it more helpful to have this statement above the burden question?

• What does “you” mean to you in the context of this question?

• If you were to replace “you” in this question, what word would you use instead?

• Would you respond to the burden question differently if “institution” replaced “you”?

Alternate Time and Burden Reporting:

• The IPEDS team at AIR has drafted an alternate time and burden question for your consideration and we would like to hear your thoughts about it. [Depending on Respondent, use either HR, Libraries, Finance, or Financial Aid as the initial hypothetical survey component]

Question 1: Were any data used in this survey component also collected for the following reporting purposes? Please mark “Yes” or “No” for each item below.

1. State required reporting …..................... □Yes □No

2. Internal institutional reporting …........... □Yes □No

3. Other required reporting ….................... □Yes □No

• In your own words, what is this question asking you?

• What does “collected” mean to you in the context of this question?

• What does “report” mean to you in the context of this question?

• What does “reporting” mean to you in the context of this question?

• What are you including when you answered [A, B, and/or C]?

Additional probes as necessary:

1. How do you gather the information required to complete the surveys?

2. How early in advance of a collection do you begin to prepare?

Question 2: Including yourself, how many staff from your institution were involved in the data collection and reporting process of this survey component?

Number of Staff

• In your own words, what is this question asking you?

• What does “staff” mean to you in the context of this question?

• What does “involved” mean to you in the context of this question?

• How did you arrive at your answer?

• Who are you including in your count?

Additional probes as necessary:

1. Do you complete the surveys yourself or are you assisted by other staff members?

2. Do you interact with any other department during the IPEDS collection?

3. Was there any particular staff member you questioned whether you should include or not? If yes, please tell me more.

Question 3. Excluding the hours spent collecting data for state and other reporting purposes, how many hours did you and others spend on each of the steps below when responding to this survey component?

Staff member	Collecting Data Needed	Revising Data to Match IPEDS Requirements	Entering Data	Revising and Locking Data
Your office	______hours	______hours	______hours	______hours
Other offices	______hours	______hours	______hours	______hours

• In your own words, what is this question asking you?

• How did you arrive at your answer?

• How certain are you of your responses?

• Who are you including in your count?

• How would your answers to the above questions differ with other components?

• What are your thoughts about these alternate time and burden questions?

• Do you feel the alternative burden questions clarify the information NCES would like you to submit in the burden question?

• How burdensome would it be to respond to the alternate time and burden questions? [Not at all burdensome, a little burdensome, somewhat burdensome, very burdensome]

Question 4. Did you need to coordinate with staff in other offices to complete this survey component?

□ Yes

□ No

Question 5. Including the time you spent before and after becoming a keyholder, how many years have you spent collecting and reporting IPEDS data?

______ Number of years

Question 6. How would you rate your knowledge and skills in the following areas?

	Mark (X) one box on each line.
Areas	Very poor	Poor	Fair	Good	Very good
Computer programming skills	□	□	□	□	□
IT Knowledge	□	□	□	□	□

Attachment IV: Handout of Test Questions

Question 1: Were any data used in this survey component also collected for the following reporting purposes? Please mark “Yes” or “No” for each item below.

1. State required reporting …..................... □Yes □No

2. Internal institutional reporting …........... □Yes □No

3. Other required reporting ….................... □Yes □No

Question 2: Including yourself, how many staff from your institution were involved in the data collection and reporting process of this survey component?

Number of Staff

Question 3. Excluding the hours spent collecting data for state and other reporting purposes, how many hours did you and others spend on each of the steps below when responding to this survey component?

Staff member	Collecting Data Needed	Revising Data to Match IPEDS Requirements	Entering Data	Revising and Locking Data
Your office	______ hours	______ hours	______ hours	______ hours
Other offices	______ hours	______ hours	______ hours	______ hours

Attachment V: Cognitive Interview Survey Items

All eleven IPEDS surveys currently ask the same burden statement (figure 1). The current burden question will be asked during the interview and the answer will be used throughout the interview to understand how the institution currently calculates time use and burden.

Figure 1. IPEDS Time and Burden Collection Question

8