This information collection request is disapproved pursuant to 5 CFR 1320.5(d)(2)(v) OMB will not approve a collection of information in connection with a statistical survey that is not designed to produce valid and reliable results that can be generalized to the universe of the study. The ICR package submitted by DOE contained four survey instruments targeted to four different potential respondent groups. These include the IAC Client Survey, the IAC Alumni Survey, the IAC Web User Survey, and the IAC Older Alumni Survey. In Part B of the supporting statement, "Collection of Information Employing Statistical Methods," DOE included a description of the methodology that would be used to implement the first three instruments listed above, however it did not describe the methodology for implementation of the IAC Older Alumni Survey. Because DOE failed to address how that particular statistical survey is designed to produce valid and reliable results that can be generalized to the universe of the study, the IAC Older Alumni Survey is disapproved. In its description of the methodology for the IAC Client Survey, DOE refers to a previous information collection, "IAC Metrics Evaluation Client Follow-up Questionnaire." OMB notes that this collection was conducted without OMB approval, in violation of the Paperwork Reduction Act. The response rate for the unapproved collection was low -- 42%. In the IAC Client Survey Portion of the ICR for which it is seeking approval, DOE fails to provide a detailed plan to improve upon the response rate and fails to demonstrate sufficiently that the design will produce valid and reliable results that can be generalized to the universe of the study. Furthermore, DOE fails to address the potential for response bias based on company size and to address how this bias may affect the representativeness of the results. Therefore, the IAC Client Survey is disapproved. DOE may wish to resubmit this portion of the proposed collection to OMB for approval, after making modifications necessary to comply with OMB guidance and 5 CFR 1320. The planned methodology for the IAC Alumni Survey is not statistically valid and will not produce results that can be generalized to the universe of the study. The chosen sampling method did not provide for random selection of participants, and the method for survey implementation will lead to an unacceptably low response rate. Therefore, the IAC_Alumni Survey is disapproved. The Web User Survey methodology also contained serious design flaws. In the proposed methodology, the respondents are self- selected and the sample size is unknown. Furthermore, similarly designed surveys have resulted in extremely low response-rate. Data from such web based surveys are only anecdotal and therefore not appropriate for producing results that can be generalized to the universe of a study. Therefore, the Web User Survey is disapproved. More broadly, the supporting statement claims the main goal of this proposed information collection is to collect data to be used in program quality monitoring. DOE fails to demonstrate how the non-generalizable information that would be generated from its proposed collection methodology would meet this goal, and thus have any practical utility. Pursuant to 5 CFR 1320.5(d)(1)(ii), an agency must demonstrate practical utility to obtain OMB approval. In addition, DOE fails to demonstrate specifically the practical utility of the data generated, including identifying actions that may be taken as a result of the information generated. OMB notes further that the survey instruments in the ICR submitted contain questions that appear to be redundant.
table that charts list comparision
Inventory as of this Action
Requested
Previously Approved
0
0
0
0
0
0
0
0
0
The Industrial Assessment Center (IAC) Program forms will survey (570) clients, alumni and web users of the IAC Program. The goal of the IAC Program, concerning this survey effort, is to develop a qualitative and quantitative understanding of how clients, alumni and Web users utilize the services and experiences pro- vided by the program and, by monitoring these activities over time, improve upon the delivery of program services.
On behalf of this Federal agency, I certify that the collection of information encompassed by this request complies with 5 CFR 1320.9 and the related provisions of 5 CFR 1320.8(b)(3).
The following is a summary of the topics, regarding the proposed collection of information, that the certification covers:
(i) Why the information is being collected;
(ii) Use of information;
(iii) Burden estimate;
(iv) Nature of response (voluntary, required for a benefit, or mandatory);
(v) Nature and extent of confidentiality; and
(vi) Need to display currently valid OMB control number;
If you are unable to certify compliance with any of these provisions, identify the item by leaving the box unchecked and explain the reason in the Supporting Statement.
Proram Quality Monitoring of IAC Participants: Clients, Alumni and Web-users