OMB Passback Q and A

interoffice memorandum

to: Shelly Wilkie Martinez

from: Ted socha

Through: kashka Kubzdela

subject: Response to 3/1/11, 3/7/11, and 3/8/11 OMB Passback on B&B:08/12 Cognitive Package

date: March 8, 2011

1. Please let us know the approximate timing of a report on this first round of cog labs and the proposed timeframe for the second set of cog labs.

A clarification, we are proposing to conduct one round of cognitive testing. Within the three week period for this round of interviews, the cog lab subcontractor will tweak question and item response option wording to test how respondents are able to comprehend and respond most appropriately to proposed new survey questions. We will submit a summary report of the cognitive testing results along with the finalized field test data collection package by 4/27.

2. Please reassure us that you have a strong basis for proposing to ask questions relating to personal stress levels (B12CEOTLN and B12GRDSTRS).  Please also tell us how you will establish the validity and reliability of such questions.

Below, we provide the rationale for each of the specific survey items mentioned above:

Question Name	B12CEOUTLN
Wording	On a scale from 1 to 10, please indicate your level of stress regarding your education-related debt.

Question Name	B12CGRDSTRS
Wording	On a scale from 0 to 10 where 0=no additional stress and 10=much more stress, please indicate the amount of stress (on top of what you already have) that you think your additional education has added/will add to your personal life.

These items are intended to get at one of the critical dimensions of the Human Capital model —the real, yet non-pecuniary cost of stress — as it is involved in the cost-benefit mix of one’s (non-)pursuit of education beyond the bachelor’s degree. Stress related to debt burden and the investments of time and effort required to pursue additional education are expected to be related to enrollment and persistence in additional education. This was a key recommendation of the B&B Technical Review Panel’s as discussed at their meeting held January 2011.

We will conduct a reliability reinterview during the field test to evaluate the temporal stability of all new items. To assess the validity of new B&B items and constructs, we propose a multistage evaluation and revision process similar to that outlined in Smith (2005)¹. Key steps in this process involve specifying expectations concerning inter-item relationships a priori, and conducting statistical analyses that evaluate how well the data match those initial expectations.

To this end, prior to field test data collection and in conjunction with NCES and with Human Capital Theory specialists, who are on the Technical Review Panel, we will compile a list of B&B items to be tested for validity and specify the expected relationships between those and other survey items.

3. For the other questions listed below, please describe plans to ensuring their validity and reliability.

Question Name	B12DFMLY1
Wording	How supportive are your parents of your current occupation?

The concept of parental support for one’s occupation is expected to be important in understanding occupational persistence. In addition to investigating the correlation between parental support and observed persistence in an occupation over the course of the longitudinal study, we will collect and code parents’ occupation and whether or not the sample member lives with his/her parents. Combining these pieces of information, we hope to be able to describe the relationship, if any, between occupational persistence and parental expectations.

Question Name	B12CELSTIN
Wording	If you were not enrolled for your [T_SL_DEGNAME], would you work for pay? How many hours per week? How much would you expect to earn [per week/month/year]?

This item provides a measure of foregone wages—an important element in the cost-benefit ratio of Human Capital Theory. To assess the convergent validity of this item, we will compare the respondents’ estimate of foregone wages with the most recent salary and work intensity (number of jobs, full-time/part-time, etc.) the respondent reported for the period prior to enrollment.

Question Name	B12FAMTRET
Wording	How much money do you have in all your retirement accounts combined?

This item is part of a larger set of questions intended to get at understanding asset formation of bachelor’s degree holders. Validity of this item will be checked by comparing aggregate statistics to those found in such studies as BLS’ National Compensation- and Current Population Surveys. Further, we will examine the relationship between this measure and related financial measures in the instrument, such as debt burden and the respondent’s previous employment history.

4. On teachers, could we ask a question about whether a current teacher underwent a traditional preparation program or an alternative route to certification?  We didn’t see this question so if it is there, what is the item number?

We included a question that has been used in SASS (the SASS:07/08 teacher survey). We added language about Teach for America to help respondents recognize alternative certification programs. B and B 2008-12 Field Test Items Cog Labs Attachment IV has been revised to include this question, and this new item is highlighted in green and can be found on page 44. The new questions will be tested as part of the cognitive interviews.

B12EALTCRT

Did you enter teaching through an alternative certification program?

(An example of an alternative program is Teach for America. It was designed to expedite the transition of non-teachers to a teaching career, for example, a state, district, or university alternative certification program.)

1 Yes

2 No

5. Generally, is there a way to ask the following question: On a scale of X (being not at all important) to Y (being highly important), how relevant do you believe the postsecondary education you just completed is to your current occupation?  We didn’t see this question so if it is there in some form, please indicate the item number.

We added the following question to the Post-Baccalaureate Employment section. B and B 2008-12 Field Test Items Cog Labs Attachment IV has been revised to include this question, and this new item is highlighted in green and can be found on page 30. This question will be asked in the job loop about each job. The new questions will be tested as part of the cognitive interviews.

B12DPBAREL

On a scale of 0 to 10, with 0 being "not at all important" and 10 being "highly important," how important do you think your [BA and all subsequent degrees] are for your job as a [FILL JOB TITLE FROM OCC]?

Spec Name	Value
Slider for each degree	0-10

6. The use of “alternative certification program” is common but incorrect because its use makes it seem that these teachers are receiving an alternative certificate versus an alternative path to getting the same certificate as all other teachers.  We would make the following change because we’re trying to move away from the use of “alternative certification program”:

B12EALTCRT

Did you enter teaching through an alternative route to certification program?

(An example of an alternative program route is Teach for America.  It was designed to expedite the transition of non-teachers to a teaching career, for example, a state, district, or university alternative route to certification program.)

1 Yes

2 No

We have made this revision in the attached B and B 2008-12 Field Test Items Cog Labs Attachment IV.

1 Smith, Gregory T. 2005. “On Construct Validity: Issues of Method and Measurement”. Psychological Assessment 17 (4): 396-408.