Response to OMB Comments

National Coordinating Center Response to 6.11.12 OMB/Weko/Mar Concerns

1. The program collection instrument asks for percentages, rather than counts (e.g. in the demography section). IPEDS has moved entirely away from collecting percentages, since this makes aggregation very difficult or impossible. Is there a point at which aggregations will be produced, rather than site-specific information? If so, then counts should be done, not percentages.

RESPONSE:

All percentages were rewritten to seek counts as recommended.

2. People at a policy level will likely want to know four or five basic things about the delivery of the TPSID program. Two commonly asked questions cannot be answered by this collection.

a. Of all who entered a TPSID program, what percentage completed?

[There is no request in here for completion rates. In general, the collection of enrollment information is weak. By convention enrollment information is collected on a census date. If that is not appropriate here, then an alternative should be proposed. And, there is no identification of a starting cohort that can be used to calculate a completion rate.]

RESPONSE:

Two clarifying questions were added to S1 to respond to this request. See S1a and S1b.

It should be noted that in EX1 and EX2 the tool captures the completion status of all students.

We will be able to calculate completion rates from the responses to the questions below. We will add S1a and S1b to the instrument to ensure we can calculate this accurately.

S1. How many students attended your TPSIDs program this year? ____

S1a. How many of the students in S1 entered your TPSID program this year? ____

S1b. How many of the students in S1 are returning TPSID students? ____

EX1. How many students exited the program this year? ____

EX2. Indicate the number of students that exited with each status below for this year. (You should account for all students in your answer for EX1)

Completed TPSID program and earned TPSID credential ___

Completed degree or certificate program available to TPSID and non-TPSID students ___

Transferred to another postsecondary education program ___

No longer wanted to attend TPSID program ___

Student was dismissed from TPSID program ___

Unknown ___

Other reason ___

b. On average, what does it cost to deliver a program like this, and how do costs vary from one program to another?

RESPONSE:

Item PD6a has been modified to provide a description of the costs to be included in the program calculation of total annual operating costs as recommended.

[Question PD6a asks: "What is the total annual operating cost for your TPSID program?" However, without providing guidance about which costs to count and how, this will generate unreliable information. If you, OPE, or they think it is important to generate this information, ]

Our expert on finance data collection in IPEDS advises that the question should ask for the program's total operating expenses. The standard definition for total annual program expenses is "the total amount spent for the activities that result in goods and services being distributed to students that fulfill the purposes or mission for which the program exists." This amount should include expenses for instruction (including faculty salaries and instructional supplies), student services, administrative support (institutional and academic support including administrative salaries and information technology), student stipends, operations and maintenance expenses, and any other expense incurred to provide program services for the year being reported.

TPSID program staff should, as needed, seek the assistance of their campus institutional research or finance office to complete this section.

3. I promised I wouldn't, but I can't help myself. There are a few glaring problems.

-- p. 16, question IIHE10. is open-ended. High burden, low usefulness

-- p. 17, question IIHE 16 asks "to the best of your knowledge have TPSID students used any of ' 18 listed IHE resources. This is not going to elicit reliable or comparable information. Asking about authorized or assisted access to IHE resources, or not asking at all, would be wiser.

RESPONSE:

Item IHE 10 was removed from the tool as recommended.

Item IHE 16 was modified to reduce response burden

Evaluation section was modified to focus on TPSID plans for follow up on exited students.

-- Evaluation activities section is burdensome and weakly constructed.

EA2., frequency of data collection [which is crossed by each type of data collection] could be eliminated. EA3. Asks "how frequently data are analyzed or reviewed." That's too vague to be useful. And if it is not useful, then it is, ipso facto, too burdensome.

RESPONSE:

We will continue to work to alleviate response burden from this section and ensure all items contribute to the CC’s evaluation of TPSID programs

-- Partnership questions are probably more granular than necessary. List all partners, then for each identify frequency of interaction, role they play, and resources they provide. That potentially a very large matrix of responses. Does ED or the US Congress need to know frequency of partner interaction, for example?

RESPONSE:

The system the TPSIDs use to report partner information is streamlined for easy entry of data. We believe the response choices available in CP2-CP4 will allow us to better characterize TPSID program operations and attributes.

A key feature of many of the TPSID programs is the partnerships and collaboration with external agencies and organizations that they have so it will be imperative that we capture who the agency is that they are partnering with and the frequency of interaction. For example, a TPSID that partners with the state VR agency on a frequent basis and is showing better student employment outcomes is important information to capture. While we cannot say that the partnership with VR caused the better employment outcomes we can say that there might be a relationship between having VR more involved with better employment outcomes and that is worthy of a more rigorous examination to determine causation.

3