December 9, 2011 Email response

Plimpton, Suzanne H.
From:
Sent:
To:
Cc:
Subject:

Earle, Janice M.
Friday, December 09, 2011 9:33 AM
Plimpton, Suzanne H.; Johnson, Roosevelt Y.; Parks, DeMonica L.
Stoel, Carol F.
FW: Revised questions from OMB

Hi Suzanne, 

Here’s the third round of questions to return to OMB on the GRF program. 

Thanks! 

janice 

From: Stoel, Carol F.
Sent: Thursday, December 08, 2011 7:11 PM
To: Earle, Janice M.; [email protected]
Cc: Lightbourne, James H.; Wilson, Valerie; [email protected]
Subject: Revised questions from OMB
Hi Janice,
NORC prepared these for us, and I checked them with Jackie Falkenheim from NCSES.
Hope this will work.
Please forward them to Suzanne Plimpton and OMB.
Hope this works.
Carol

From: Jake Bartolone [mailto:[email protected]]
Sent: Thu 12/8/2011 3:08 PM
To: Stoel, Carol F.; Marie Halverson
Cc: Sheila Nataraj Kirby; Tom Hoffer; Gregory Wolniak; Lisa Setlak; Melissa Yisak; Muller-Parker, Gisele T.; John, Gilbert;
Johnson, Roosevelt Y.
Subject: RE: last few points from OMB

Hi Carol,
As requested, here are the final questions as they will appear in our questionnaire (i.e., without editing notes as 
displayed below).
Thanks, and please let me know if you have any further questions.

Jake

2.            Which of the following best describes your ethnicity?
             Hispanic or Latino (go to Question G.3)
             Not Hispanic or Latino (go to Question G.4)
3.            Which of the following best describes your Hispanic or Latino origin or descent?
Mark one.
1

             Mexican or Chicano
             Puerto Rican
             Cuban
             Other Hispanic
4.            Do you consider yourself (select one or more)…
             American Indian or Alaska Native
             Asian
             Black or African American
             Native Hawaiian or Other Pacific Islander
             White

10.
Are you deaf or do you have serious difficulty hearing?
Are you blind or do you have serious difficulty seeing even when wearing glasses?
Because of a physical, mental, or emotional condition, do you have serious difficulty concentrating, remembering, or 
making decisions?
Do you have serious difficulty walking or climbing stairs?
Do you have difficulty dressing or bathing?
Because of a physical, mental, or emotional condition, do you have difficulty doing errands alone such as visiting a 
doctor’s office or shopping?

Jake Bartolone, Ph.D. | Research Scientist
NORC at the University of Chicago
55 East Monroe Street, 30th Floor, Chicago IL 60603
[email protected] | office (312) 759-4002

From: Jake Bartolone
Sent: Wednesday, December 07, 2011 4:34 PM
To: 'Stoel, Carol F.'; Marie Halverson
Cc: Sheila Nataraj Kirby; Tom Hoffer; Gregory Wolniak; Lisa Setlak; Melissa Yisak; 'Muller-Parker, Gisele T.'; 'John,
Gilbert'; 'Johnson, Roosevelt Y.'
Subject: RE: last few points from OMB

Hi Carol,
Here are our revised questions for the two OMB comments that you sent. For the race and ethnicity questions, OMB 
would like us to adapt the OMB standards. For the disability questions, they would like us to adopt the categories used 
in the American Community Survey (ACS). I’ve pulled the ACS questions for both race/ethnicity and disability and put 
them next to the current GRFP versions below. However, I do not believe that they want us to adopt the ACS 
race/ethnicity questions, but rather to alter our current version to remove the “two or more races” category, and 
instead allow multiple selections to that question. The OMB standards in question read:
‐When self‐identification is used, a method for reporting more than one race should be adopted.
2

‐The method for respondents to report more than one race should take the form of multiple responses to a single 
question and not a "multiracial" category.
‐When a list of races is provided to respondents, the list should not contain a "multiracial" category.
‐Based on research conducted so far, two recommended forms for the instruction accompanying the multiple response 
question are "Mark one or more ..." and "Select one or more...."
(Source: OMB website on Revisions to the Standards for the Classification of Federal Data on Race and Ethnicity, 
 http://www.census.gov/population/www/socdemo/race/Ombdir15.html)
I’ve included below a revised version of the GRFP question that hews to these standards – I believe that we should use 
this version, although I’ve included the ACS version (which has many more categories) for reference. Note that the OMB 
standards also include changing “Alaskan” to “Alaska Native”, which I have incorporated below.
To be clear, I’ve highlighted the version of each question that we propose to use in the GRFP survey. Please let me know 
if you have any questions or if you’d like for us to send this information in another format.
Thanks,
Jake
This is the current GRFP set of questions on race and ethnicity:
2.            Which of the following best describes your ethnicity?
             Hispanic or Latino (go to Question G.3)
             Not Hispanic or Latino (go to Question G.4)
3.            Which of the following best describes your Hispanic or Latino origin or descent?
Mark one.
             Mexican or Chicano
             Puerto Rican
             Cuban
             Other Hispanic
4.            Do you consider yourself (choose one)…
             American Indian or Alaskan
             Asian
             Black or African American
             Native Hawaiian or Other Pacific Islander
             White
             Two or more races
4a. If two or more races, please select the racial groups to which you belong (choose more than one)…
             American Indian or Alaskan
             Asian
             Black or African American
             Native Hawaiian or Other Pacific Islander
             White
             Two or more races
This is a revised version of the GRFP set of questions on race and ethnicity that conforms to OMB standards (changes are 
in red):
2.            Which of the following best describes your ethnicity?
             Hispanic or Latino (go to Question G.3)
             Not Hispanic or Latino (go to Question G.4)
3

3.            Which of the following best describes your Hispanic or Latino origin or descent?
Mark one.
             Mexican or Chicano
             Puerto Rican
             Cuban
             Other Hispanic
4.            Do you consider yourself (select one or more)…
             American Indian or Alaska Native
             Asian
             Black or African American
             Native Hawaiian or Other Pacific Islander
             White
             Two or more races
4a. If two or more races, please select the racial groups to which you belong (choose more than one)…
             American Indian or Alaskan
             Asian
             Black or African American
             Native Hawaiian or Other Pacific Islander
             White
             Two or more races

This is the ACS version for reference only (edited to change references from “this person” to “you”):
Are you of Hispanic, Latino, or Spanish origin?
No, not of Hispanic, Latino, or Spanish origin
Yes, Mexican, Mexican Am., Chicano
Yes, Puerto Rican
Yes, Cuban
Yes, another Hispanic, Latino, or Spanish origin – Print origin, for example, Argentinian, Colombian, Dominican, 
Nicaraguan, Salvadoran, Spaniard, and so on.
What is your race? Mark (X) one or more boxes.
White
Black, African Am., or Negro
American Indian or Alaska Native – Print name of enrolled or principal tribe.
Asian Indian
Japanese
Native Hawaiian
Chinese
Korean
Guamanian or Chamorro
Filipino
Vietnamese
Samoan
Other Asian – Print race, for example, Hmong, Laotian, Thai, Pakistani, Cambodian, and so on.
Other Pacific Islander – Print race, for example, Fijian, Tongan, and so on.
Some other race – Print race.

4

This is the current GRFP set of questions on disability:
10.          Do you have the following disabilities? Please mark Yes or No for each. 1
Yes         No
□             □             Hearing impairment
□             □             Visual impairment
□             □             Mobility/orthopedic impairment
□             □             Learning/Cognitive Disability
□             □             Vocal/Speech Disability
□             □             Other (Please specify) ______________________
This is the ACS version (edited to change references from “this person” to “you”):
Are you deaf or do you have serious difficulty hearing?
Are you blind or do you have serious difficulty seeing even when wearing glasses?
Because of a physical, mental, or emotional condition, do you have serious difficulty concentrating, remembering, or 
making decisions?
Do you have serious difficulty walking or climbing stairs?
Do you have difficulty dressing or bathing?
Because of a physical, mental, or emotional condition, do you have difficulty doing errands alone such as visiting a 
doctor’s office or shopping?

Jake Bartolone, Ph.D. | Research Scientist
NORC at the University of Chicago
55 East Monroe Street, 30th Floor, Chicago IL 60603
[email protected] | office (312) 759-4002

From: Stoel, Carol F. [mailto:[email protected]]
Sent: Tuesday, December 06, 2011 9:57 AM
To: Marie Halverson
Cc: Sheila Nataraj Kirby; Tom Hoffer; Jake Bartolone; Gregory Wolniak; Lisa Setlak; Melissa Yisak; Muller-Parker, Gisele
T.; John, Gilbert; Johnson, Roosevelt Y.
Subject: last few points from OMB

HI Everyone,
Just two little comments from OMB which I believe NORC folks can fix easily.
If you cannot find the references mentioned, please let us know.
As soon as you can turn it around, please send it back to us and we’ll return it to OMB.
I can almost taste closure…
Thanks,
Carol
FROM OMB

We are happy with the changes made. However, there are two issues that remain in the survey instrument:
‐
‐

Question G4 on the demographic section uses the wrong race format.  There should not be a “two or more” 
category, and the prompt should comply with the OMB standard. 
Question G10‐ recommend changing the disability questions to match the prompt and answer categories 
on the American Community Survey, which provides a much more reliable measure of disability. 
5

Thanks,

From: Jake Bartolone [mailto:[email protected]]
Sent: Monday, December 05, 2011 8:34 PM
To: Muller-Parker, Gisele T.; Stoel, Carol F.; John, Gilbert
Cc: Marie Halverson; Sheila Nataraj Kirby; Tom Hoffer; Gregory Wolniak; Lisa Setlak; Melissa Yisak
Subject: RE: Next steps for GRFP federal program review

Thanks, Gisele. It's a lot to review -- if we were to meet early next week (Monday or Tuesday) would that give you enough time back
in the office to review everything? I'm fine with meeting on the 15th if not.
Thanks again,
Jake

-----Original Message----From: Muller-Parker, Gisele T. [mailto:[email protected]]
Sent: Mon 12/5/2011 6:12 PM
To: Jake Bartolone; Stoel, Carol F.; John, Gilbert
Cc: Marie Halverson; Sheila Nataraj Kirby; Tom Hoffer; Gregory Wolniak; Lisa Setlak; Melissa Yisak
Subject: RE: Next steps for GRFP federal program review
Jake,

This is great. I look forward to reviewing the documents. So far, the
responses I have received are that the interviewees will be the members
of the Inter-agency group. I agree that I should make the initial
contact.

I'm sorry but Carol Stoel and I will be at the Council of Graduate
Schools meeting in Arizona for the rest of this week. I can meet next
week, Monday through Thursday. Is it best if we use our 3 pm Thursday
time slot on Dec 15? If you want to meet earlier, I am available as
early as Monday.

Gisele
Gisele Muller-Parker, PhD
Program Director, Division of Graduate Education
National Science Foundation
4201 Wilson Blvd. Suite 875 S
Arlington, VA 22230
Phone: (703) 292-7468
Fax: (703) 292-9048

From: Jake Bartolone [mailto:[email protected]]
Sent: Monday, December 05, 2011 5:50 PM
To: Muller-Parker, Gisele T.; Stoel, Carol F.; John, Gilbert
Cc: Marie Halverson; Sheila Nataraj Kirby; Tom Hoffer; Gregory Wolniak;
6

Lisa Setlak; Melissa Yisak
Subject: Next steps for GRFP federal program review

Hi Gisele, Carol, and Gilbert,

In order to keep things moving, I've attached a number of documents
related to the federal program review that we'll need to finalize before
moving forward. These are:

1) A draft interview protocol in the format we will be using to conduct
interviews (program review interview protocol with intro edited.docx) this shows the introduction and wrap-up as well as the question
formulation and order that we plan to use. We will skip questions if we
are able to find the relevant information beforehand, and I've made a
preliminary attempt a prioritization that I would also like your
feedback on. I've highlighted what I believe to be the most important
questions in red (note that we may be answer some of the prioritized
questions with publicly available information).
2) Drafts for two different emails to program officers (federal program
review draft contact emails Dec 5.docx) - the first to be sent by
Gisele, the second by me. The first email draft is a suggestion only, as
it will be coming from NSF rather than NORC, but includes the type of
information we would prefer be included (such as my contact information
and a request for any program materials that may help us answer the
questions prior to the interview).
3) The current list of programs to include (List of programs to be
reviewed.docx), which has remained unchanged for a while now.
4) Two examples of information that we were able to find by reviewing
program websites (Program Review Matrix--two examples, Nov 21 2011.docx)
- Sheila filled out our matrix for the Department of Defense SMART
program and the Department of Energy CSGF program. We note that there's
a broad range in the amount of available information, and some programs
have far less information available than these two, but that this should
illustrate the types of information we are likely to be able to find
prior to the interviews.

In addition to the attached documents, we'll need to discuss and decide
upon a schedule for sending the initial contact email, conducting
interviews, and delivering the report. I'd also like to briefly touch
base about how we plan to move forward with selecting institutions for
the site visits and institutional phone interview components that are
coming up, so that we can start preparing for that work as well.

Would it be possible to meet later this week to discuss the attached
documents and talk about our schedule moving forward?

7

Thanks,

Jake

Jake Bartolone, Ph.D. | Research Scientist
NORC at the University of Chicago
55 East Monroe Street, 30th Floor, Chicago IL 60603
[email protected] | office (312)
759-4002

8