2019-20 National Postsecondary Student Aid Study (NPSAS:20) Pretesting

National Center for Education Statistics (NCES)

Attachment IX

Student Opinion Survey Using Crowdsourcing Platform

2019-20 National Postsecondary Student Aid Study (NPSAS:20) Pretesting

OMB# 1850-0803 v. 243

December 2018

This attachment provides the platform advertisement, a list of the items to be included in the 2019-20 National Postsecondary Student Aid Study (NPSAS:20) student opinion survey, and survey item specifications.

Table 1 provides a summary of the content in the student opinion survey. Each item is listed by section name, form name, and form label.

Platform advertisement

Survey description: This survey is about what motivates postsecondary students to participate in surveys.

Time allotted: 15 minutes

Expires: 04/30/2019

Qualifications required: Resides in U.S. and enrolled in postsecondary education

Figure 1. Example advertisement from a previous study

Table 1. List of all student opinion survey forms

Section name	Form name	Label
Introduction	INFO	Study information
Eligibility	N20AINTROMTurk	Survey introduction form
Eligibility	ELIGHS	Attending high school in 2018-19 academic year
Eligibility	ELIGCOL	Attending college, university, or trade school in 2018-19 academic year
Eligibility	ELIGAGE	Age ranges
Eligibility	TERMIN	Survey termination for ineligible respondents
Incentives	INCMOT	Incentives: types of incentives to motivate participation
Incentives	INCAMT	Incentives: reasonable amount of money
Incentives	INCPREF	Incentives: preferred method of monetary incentive
Incentives	INCNONM	Incentives: type of gift or service would like to receive
Incentives	INCNONWHY	Incentives: why most preferred gift or service incentive
Incentives	INCNONPREF	Incentives: prefer gift or service over monetary incentive
Incentives	INCNONWHAT2	Incentives: type of gift or service over monetary incentive
Incentives	INCNONSERV	Incentives: already use gift or service
Incentives	INCORDER	Incentives: gift or service order issue contact
Incentives	INCORDER2	Incentives: gift or service product issue contact
Contact Methods	CONBEST	Contacting: best method for contacting about survey
Contact Methods	CONTEXT	Contacting: how react to receiving text message about survey
Contact Methods	EMLSPAM	E-mail: legitimacy features
Contact Methods	EMLUNSUB	E-mail: unsubscribe link
Contact Methods	EMLHTML	E-mail: HTML or plain text preference
Contact Methods	EMLHTMLWHY	E-mail: reason for HTML preference
Contact Methods	EMLHTMLDIF	E-mail: difficulty viewing HTML e-mails
Contacting Materials	ENVINTRO	Introduction to contacting materials
Contacting Materials	ENVRANK	Envelopes: ranking designs
Contacting Materials	ENVWHY	Envelopes: most important factor in preferred envelope
Contacting Materials	ENVOPEN	Envelopes: likeliness of opening preferred envelope
Contacting Materials	ENV2DOL	Envelopes: importance of $2 bill image
Contacting Materials	BROCHURE	Brochure information
Contacting Materials	BROTOPIC	Brochures: important topics to include
Contacting Materials	BRORANK	Brochures: ranking topics to include
Contacting Materials	BROREAD	Brochures: likeliness of reading
NPSAS Survey Items	N20SVYINTRO	Introduction to NPSAS:20 survey items
NPSAS Survey Items	N20ASX	Sex assigned at birth
NPSAS Survey Items	N20AMARR	Marital status
NPSAS Survey Items	N20EINCOM	Income for prior calendar year from all sources
NPSAS Survey Items	N20ANPSAS	Name of NPSAS institution
NPSAS Survey Items	N20ACURENR	Current attendance at NPSAS institution
NPSAS Survey Items	N20ADEGREE	Degree program at NPSAS institution
NPSAS Survey Items	N20ADIPL	Type of high school completion
NPSAS Survey Items	N20APRDG	Earned prior degree or certificate
NPSAS Survey Items	N20ANFST	NPSAS institution was first postsecondary institution attended after high school
NPSAS Survey Items	N20CAIDGATE	Financial aid gate for the 2018-19 academic year
NPSAS Survey Items	N20CFEDLN	Took out any federal student loans in the 2018-19 academic year
NPSAS Survey Items	N20CPRVLN	Took out any private student loans in the 2018-19 academic year
NPSAS Survey Items	N20DANYJOB	Worked for pay at any time in the 2018-19 academic year
End	N20ENDB	Additional comments about survey experience
End	END	End form

Contents

Student Opinion Survey Item Specifications 6

INFO 6

N20AINTROMTurk 6

ELIGHS 6

ELIGCOL 6

ELIGAGE 6

TERMIN 6

INCMOT 7

INCAMT 7

INCPREF 7

INCNONM 7

INCNONWHY 7

INCNONPREF 7

INCNONWHAT2 7

INCNONSERV 8

INCORDER 8

INCORDER2 8

CONBEST 8

CONTEXT 8

EMLSPAM 9

EMLUNSUB 9

EMLHTML 9

EMLHTMLWHY 9

EMLHTMLDIF 9

ENVINTRO 9

ENVRANK 10

ENVWHY 10

ENVOPEN 11

ENV2DOL 11

BROCHURE 11

BROTOPIC 12

BRORANK 12

BROREAD 13

N20SVYINTRO 13

N20ASX 13

N20AMARR 13

N20EINCOM 13

N20ANPSAS 14

N20ACURENR 14

N20ADEGREE 14

N20ADIPL 14

N20APRDG 14

N20ANFST 15

N20CAIDGATE 15

N20CFEDLN 15

N20CPRVLN 15

N20DANYJOB 15

N20ENDB 16

END 16

Student Opinion Survey Item Specifications

INFO

This survey is conducted by the National Center for Education Statistics (NCES) within the U.S. Department of Education. The survey will take approximately 15 minutes to complete. Your participation is voluntary, and you have the right to stop at any time.

This survey is being administered by RTI International under OMB No. 1850-0803 and resides on a server outside of the NCES domain. NCES cannot guarantee the protection of survey responses and advises against the inclusion of sensitive personal information in any response. By proceeding, you give your consent to participate in this survey.

Only eligible respondents who complete this survey will be compensated. Please take your time as you answer these questions. The information you provide will contribute to valuable research at NCES.

N20AINTROMTurk

Thank you for your interest in this survey. Eligible respondents will help us develop strategies that will attract people like you to become future survey respondents.

To begin, we have a few questions that will help us determine your eligibility for this study.

ELIGHS

Were you attending high school or completing any high school requirements at any time between July 1, 2018 and today?

1 = Yes (Ineligible)

0 = No

ELIGCOL

Did you attend a college, university, or trade school at any time between July 1, 2018 and today?

1 = Yes

0 = No (Ineligible)

ELIGAGE

What is your age?

1 = Under 18 (Ineligible)

2 = 18-24

3 = 25-29

4 = 30-34

5 = 35-39

6 = 40-49

7 = 50-59

8 = 60 or older

TERMIN

Thank you for your expressed interest. Based on the requirements of this study, we are not able to include you in the study at this time.

INCMOT

Usually, respondents receive something as a token of appreciation (an incentive) for completing a survey. What kind of incentive would motivate you to complete a 30-minute survey on questions around costs and benefits of postsecondary education?

INCAMT

If you were to receive a monetary incentive for completing a 30-minute survey, how much money do you think is reasonable?

$ | .00

INCPREF

What would be your preferred method to receive such a monetary incentive?

1 = Check

2 = PayPal

3 = Prepaid Credit Card

4 = Other

Please specify: ___________________________

INCNONM

Which of the following non-monetary incentives, such as a gift or a service, would you most like to receive to complete a 30-minute survey?

1 = a snack or food delivery subscription (such as Graze, HelloFresh, Blue Apron)

2 = an entertainment subscription (such as Netflix, Hulu, Spotify)

3 = a service subscription (such as Lifelock, LinkedIn Premium)

4 = an online book rental subscription (such as Chegg, Kindle)

5 = an online shopping subscription offering free shipping and/or free entertainment such as music and video (such as Amazon Prime)

6 = an online catalog with multiple items from which to choose

7 = something else

Please specify: ___________________________

INCNONWHY

Why did you select [INCNONM] as your most preferred non-monetary incentive?

INCNONPREF

Would you rather receive [INCNONM] or money as an incentive for completing a survey, even if the amount of money offered is less than the monetary value of [INCNONM]?

1 = I would prefer [INCNONM].

2 = I would prefer money, even if the amount is less than the value of [INCNONM].

INCNONWHAT2

What types of non-monetary incentives would be better to receive than money?

INCNONSERV

On a scale from 1 to 5 with 1 being "Completely likely" and 5 being "Not at all likely,” please indicate how likely it would be for you to complete a survey if a service, such as a book rental or entertainment subscription, that you already use was offered as an incentive?

1 = Completely likely

2 = Very likely

3 = Moderately likely

4 = Slightly likely

5 = Not at all likely

INCORDER

Now imagine you were offered a non-monetary incentive and you decided to complete the survey. If something were to happen to your non-monetary incentive order, such as you never received the order, or there were issues with the website, who would you contact first?

1 = The vendor of the incentive (e.g., Spotify or another subscription or service provider)

2 = The agency conducting the survey (e.g., U.S. Department of Education)

3 = The survey contractor (e.g., RTI International or another research organization)

4 = Other

Please specify: ___________________________

INCORDER2

Please imagine you had issues with the non-monetary incentive product you received. Who would you contact about those issues?

1 = The vendor of the incentive (e.g., Spotify or another subscription or service provider)

2 = The agency conducting the survey (e.g., U.S. Department of Education)

3 = The survey contractor (e.g., RTI International or another research organization)

4 = Other

Please specify: ___________________________

CONBEST

What is the best method to contact you about a survey?

1 = E-mail

2 = Letter or postcard in the mail

3 = Call to home phone

4 = Call to cell phone

5 = Text message

6 = Other

Please specify: ___________________________

CONTEXT

Often, during a study, individuals receive an invitation letter in the mail and reminders to complete the survey via text message or e-mail.

How would you react if you were to receive a text message reminder to complete a survey? The text message would include a link to open the survey on your phone.

Please select all that apply.

I would ignore it

I would block the text message sender from my phone

I would report the text message sender as spam

I would be upset that I have to pay for unsolicited texts

I would use the provided link to complete the survey

Other

Please specify: ___________________________

EMLSPAM

When you receive an e-mail from a new sender, what features do you typically look for to decide if the e-mail is legitimate or not?

EMLUNSUB

Would you expect an "unsubscribe" link in e-mails asking you to complete a survey from or on behalf of a governmental agency?

1 = Yes

0 = No

EMLHTML

HTML e-mails are those with graphic designs/colors and plain text e-mails contain just text. Do you prefer to receive HTML e-mails or plain text e-mails?

1 = I prefer HTML

2 = I prefer plain text

EMLHTMLWHY

Why do you prefer HTML e-mails?

EMLHTMLDIF

Do you ever have difficulty viewing HTML e-mails?

1 = Yes

0 = No

-1 = Don’t know

ENVINTRO

Now we would like to ask a few questions about the mailing materials you might receive inviting you to complete a survey.

ENVRANK

Please rank each of the four envelopes from 1 to 4 with 1 being the envelope you are most likely to open and 4 being the envelope you are least likely to open.

ENVWHY

Consider the envelope that you ranked the highest, that is, the one that you are most likely to open. What was the most important factor that influenced your decision to rank this envelope as the one you are most likely to open?

ENVOPEN

Again, thinking of the envelope that you ranked as most likely to open. How likely are you to actually open this envelope?

1 = Completely likely

2 = Very likely

3 = Moderately likely

4 = Slightly likely

5 = Not at all likely

ENV2DOL

Now, we would like to focus on the envelope with the $2 bill, below.

How important was the picture of the $2 bill in your ranking of the envelopes?

1 = Extremely important

2 = Very important

3 = Moderately important

4 = Slightly important

5 = Not at all important

In what way, if any, did the suggested or actual value of the incentive influence your ranking of the envelope with the picture of the $2 bill?

BROCHURE

Imagine that you received an envelope in the mail, addressed to you, containing an invitation letter and a study brochure or a flyer. The invitation letter typically contains information on the study, how you were selected, what incentive is offered (or already included), and how you can log in to take the survey.

Should the brochure convey only new information, or should we repeat some of the topics also included in the invitation letter? Which, if any, topics should we repeat?

BROTOPIC

Below are examples of study brochures that provide individuals with background information on a study and a list of topics that are often included in study brochures. Please select a maximum of three topics that you think should be included in a study brochure or pamphlet.

Study content

Types of questions

Purpose of the study

The agency conducting the survey

Introducing the study director and team

How people are selected for the study

Use of respondent’s data

Privacy and confidentiality

Individual’s rights (voluntary participation, etc.)

Study contact/help desk information

Past results of the same study

Other

Please specify: _________________

BRORANK

You just selected three topics to include in a brochure. Please rank the three topics you just identified in order of importance with 1 being the topic that you find most important to include and 3 being the topic that you find least important to include.

[Topic 1]

[Topic 2]

[Topic 3]

BROREAD

How likely are you to actually read a brochure or pamphlet that you receive in the mail?

1 = Completely likely

2 = Very likely

3 = Moderately likely

4 = Slightly likely

5 = Not at all likely

N20SVYINTRO

Finally, to better understand the backgrounds of those completing this survey, we have a few questions about you and your education experiences.

N20ASX

What sex were you assigned at birth (what the doctor put on your birth certificate)?

1 = Male

2 = Female

N20AMARR

What is your current marital status?

1 = Single, never married

2 = Married

3 = Separated

4 = Divorced

5 = Widowed

6 = Living with a partner in a marriage-like relationship

N20EINCOM

What was your income for calendar year 2018, prior to taxes and deductions?

(Calendar year 2018 includes January 1, 2018 through December 31, 2018. Include all income you paid taxes on, including work, investment income, or alimony. Do not include income from a spouse. Do not include any grants or loans you may have used to pay for school, or any money given to you by your family.)

No income

Less than $5,000

$5,000 - $9,999

$10,000 - $14,999

$15,000 - $19,999

$20,000 - $29,999

$30,000 - $39,999

$40,000 - $49,999

$50,000 - $74,999

$75,000 or more

N20ANPSAS

What is the name of the college, university, or trade school that you have attended at any time between July 1, 2018 and today?

(If you have attended more than one college, university, or trade school, please provide the name of the school you consider to be your primary school.)

N20ACURENR

Are you currently attending [NPSAS institution]?

1 = Yes

0 = No

N20ADEGREE

What degree or certificate were you working on during your most recent term of enrollment at [NPSAS institution] in the 2018-2019 academic year (July 1, 2018 - June 30, 2019)?

Associate’s degree (usually a 2-year degree)

Bachelor’s degree (usually a 4-year degree)

Master’s degree

Doctoral degree--research/scholarship (for example, PhD, EdD, etc.)

Doctoral degree--professional practice (including: chiropractic, dentistry, law, medicine, optometry, pharmacy, podiatry, or veterinary medicine)

Doctoral degree—other

Undergraduate certificate or diploma (usually less than 2 years), including those leading to a certification or license (example: cosmetology)

Post-baccalaureate certificate

Post-master's certificate

Undergraduate level classes

Graduate level classes

N20ADIPL

Which of the following best describes your high school completion?

1 = Received a high school diploma in the United States

2 = Received a GED (General Educational Development) certificate or other equivalent credential (such as HiSET or TASC)

3 = Received a high school completion certificate

4 = Attended a foreign high school

6 = Home schooled

5 = Did not complete high school or a high school equivalent

N20APRDG

Have you earned any degrees or certificates since you completed your high school requirements?

(Only include degrees or certificates earned through a college, university, or trade school.)

1 = Yes

0 = No

-1 = Don’t know

N20ANFST

Was [NPSAS institution] the first college, university, or trade school you attended after completing your high school requirements?

1 = Yes

0 = No

N20CAIDGATE

Next, we have some questions about how you paid for your education.

Please indicate whether or not you received or used any of the following to pay for your education in the 2018-2019 academic year. Check all that apply.

Loans (from any source)

Grants or scholarships from the government or [N20ANPSAS]

Earnings from employment (other than sources listed here)

Work-study (i.e., federal, state, or institution work-study)

Financial assistance from your [{if N20AMARR = 2} spouse,] parents or guardians, friends, or family

Assistantship(s)

Financial assistance from your employer or from your parents’ or guardians’ employers

Scholarship(s) from a private organization

Veterans education benefits

Fellowship(s)

Internship, clinical residency

Traineeship

Foreign (non-U.S.) support

Other source

1 = Yes

0 = No

N20CFEDLN

Thinking only about the 2018-2019 academic year, did you take out any federal student loans?

(Here federal student loans include subsidized and unsubsidized Direct Loans [also known as Stafford Loans], Perkins Loans, and graduate student PLUS Loans, but do not include parent PLUS Loans.)

1 = Yes

0 = No

N20CPRVLN

Thinking only about the 2018-2019 academic year, did you take out any private loans borrowed from a private lender?

(Private loans are borrowed from a private lender such as a bank or sometimes a state, usually require a co-signer, and have market interest rates based on credit history. Click here for examples of private loans).

1 = Yes

0 = No

N20DANYJOB

Have you worked for pay at any time between July 1, 2018 and today?

1 = Yes

0 = No

N20ENDB

If you have any additional comments about your overall experience participating in this survey, please provide them now.

END

Thank you for your time and cooperation. We greatly appreciate your assistance.