Supporting Statement
Weatherization Assistance Program Evaluation
OMB Control Number: (NEW)
As a pre-amble to this supporting statement, the U.S. Department of Energy (DOE) and Oak Ridge National Laboratory (ORNL) are requesting an emergency review of this supporting statement and attached data collection instruments and surveys related to evaluating the U.S. Department of Energy’s Weatherization Assistance Program (WAP). The purpose of this Program is to weatherize low-income homes throughout the United States. In normal program years, WAP receives approximately $250 million in annual appropriations. The American Recovery and Reinvestment Act (ARRA) contain $5 billion dollars for this program. The expectation is that with this enhanced funding the Program will weatherize one million homes per ARRA year and spur job creation.
With this background in mind, two major, inter-related issues are cited to support this emergency request. The first is related to ARRA. The Administration requires that expenditure of these funds be closely tracked and programmatic outcomes achieved with the funds be rigorously documented. The Administration also desires that ARRA funds be spent as expeditiously as possible. This project is being supported with ARRA funds (DOE recently obligated $5.8 million of ARRA funds to ORNL to take the project through FY’s 09 and 10). As outlined below, data collected by this project will provide a national energy savings baseline with which to compare WAP ARRA program year’s savings and several special energy savings studies that are part of this project, and will help to identify energy savings in ARRA WAP program year 2010. Thus, this project is intimately tied to ARRA and needs to begin as quickly as possible.
The second issue is that delays in initiating the data collections described in this document threaten the ability to create the national energy savings baseline and the conduct of the special studies. With respect to the first point, the evaluation plan calls for ORNL and its independent subcontractor to collect billing histories from utility companies for WAP program years 2007 and 2008. Substantial delays in data collection will threaten the ability to collect 2007 data from the utilities because the utilities routinely discard the data after a couple of years. Losing access to 2007 data from the utilities will severely impair the ability to create the national energy savings baseline needed to assess WAP ARRA years performance. With respect to the second point, the evaluation plan contains special studies to estimate energy savings in weatherized homes that heat with fuel oil and propane, and weatherized mobile and multi-family homes. Data for these special studies will be related to WAP ARRA program year 2010 weatherization activity. Procedures to collect the required data for these special studies need to be in place well before the winter of 2010/2011 so that appropriate pre and post weatherization data can be collected. Backing up from this requirement, then, the project will need to be able to be in the field with appropriate supporting surveys by March 15, 2010. Thus, we are requesting emergency approval by this date.
A. Justification
Explain the circumstances that make the collection of information necessary. Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the information collection.
The collection of this information is necessary in order to verify the cost-effective performance of authorized tasks by the Department of Energy’s Weatherization Assistance Program, highlight best practices in reaching program objectives, and identify ways in which program performance can be improved. In its PART (Program Assessment Rating Tool) rating for the program, the Office of Management and Budget (OMB) found that:
“The program lacks an assessment of performance that is current, comprehensive, and independent. The program reports a favorable benefit-cost ratio through internal assessments. However, these assessments rely in part on old data…”
The last comprehensive evaluation of the program was conducted in 1993 and employed data from the 1989-1990 program year (PY). Many aspects of the program’s rules, regulations, administrative structures and program operations have changed since that time and more current statistical assessments are limited in scope and tied to the original evaluation data base.
Authority to conduct survey and evaluation activity can be found in 10 CFR 440, Section 440.23(a) mandating that the Secretary of Energy shall “…monitor and evaluate the operation of projects carried out by CAA’s [community action agencies] receiving financial assistance under this part through on-site inspections, or through other means, in order to ensure the effective provision of weatherization assistance for the dwelling units of low-income persons.”
Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection.
The information that will be gathered and analyzed through this collection will have multiple audiences. It will be used to inform Congress, the Department, and the administration of the current state of program performance. Statistics will be used to update and improve PART and GPRA (Government Performance and Results Act) assessments. Results of the study will also be used to identify strengths and weaknesses of program performance in order to target ways in which this can be improved at federal, state, and local implementation levels.
In addition, findings from the previous evaluation that program performance in hot-climate regions was not on par with the national average, the impetus for a hot-climate initiative, which has targeted that region with additional training, technical assistance, and unique weatherization techniques in order to improve performance. It is anticipated that findings from the current evaluation will result in similar adjustments and improvements to the Weatherization Assistance Program.
Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses.
It is expected that over 90% of the data collection effort for this study will involve some form of electronic exchange, either through the use of online survey instruments or the transfer of utility billing records or other relevant household data via electronic mail and attached files.
Requesting utility bills and other relevant household data in electronic format will ease the reporting burden of the utilities and local agencies providing the information and will yield information in a form that is easier for the evaluation contractor to process. This approach is expected to reduce the time required by all parties.
Describe efforts to identify duplication.
There is no similar information available on a national basis. In fact, there is no program comparable in scale or scope to the Weatherization Assistance Program and only a comprehensive national evaluation can now provide an accurate picture of this program’s operation and performance.
If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.
The information collection is not expected to have a significant economic impact on small businesses or other small entities.
Describe the consequence to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.
If the data collection is not allowed to proceed, the program’s performance will be increasingly in doubt for policy makers. Most important, managers at the federal, state, and local levels will be deprived of essential information regarding program operations and areas in need of improvement. This information ranges from appropriate client education techniques, best audit methods, and crew training to energy efficiency measures selection and appropriate health and safety techniques for clients and crews. In short, the information is vital to insure that the program delivers effective services to low-income households with hundreds of millions of federal dollars per annum nearly two decades after the last national program assessment was performed.
Explain any special circumstances that require the collection to be conducted in a manner inconsistent with OMB guidelines. (a) requiring respondents to report information to the agency more often than quarterly; (b) requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it; (c) requiring respondents to submit more than an original and two copies of any document; (d) requiring respondents to retain records, other than health, medical government contract, grant-in-aid, or tax records, for more than three years; (e) in connection with a statistical survey, that is not designed to product valid and reliable results that can be generalized to the universe of study; (f) requiring the use of statistical data classification that has not been reviewed and approved by OMB; (g) that includes a pledge of confidentially that is not supported by authority established in stature of regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; (h) requiring respondents to submit proprietary trade secrets, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information’s confidentiality to the extent permitted by law.
This information collection will not involve any of the special circumstances itemized in the Specific Instructions for this Supporting Statement.
If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the agency’s notice, required by 5CFR 320.8(d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken in response to the comments. Specifically address comments received on cost and hour burden. Describe efforts to consult with persons outside DOE to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or report.
DOE published in the Federal Register on January 12, 2007 (Volume 72, Number 8) on pages 1495-1496 a 60-day notice as required by 5CFR1320.8(d) to solicit comments on the information collection prior to submission to OMB. No comments were received in response to this notice, but there was one request for the survey instruments. The requester was directed to the Preliminary Evaluation Plan posted on an Oak Ridge National Laboratory (ORNL) web page, which contains draft versions of the survey instruments to be used in the data collection.
DOE also published in the Federal Register on March 17, 2010 (Volume 75, Number 51) on pages 12736-12737 a 14-day notice to solicit comments on this emergency information collection. No comments were received in response to this notice.
Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.
No payments will be made to state and local agency respondents for information collection instruments used in this study. Similarly, no payments will be made to those local agencies and utility companies who will be asked for information related to billing records, metering data, and subject identification.
Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy.
The information provided by respondents to the surveys and household data requests will be reported only in the aggregate and a subject’s name, agency, or other identifying information will not be reported in association with the individual answers. The evaluation contractor selected to perform this study will be required to take appropriate steps to protect the privacy of those agencies and individuals from whom data are collected. A key component of this is identifying and addressing potential vulnerabilities in the processes by which electronic data are collected and stored.
Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private. This justification should include the reasons why DOE considers the questions necessary, the specific uses to be made of the information., the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.
No questions of a sensitive nature are included in any of the surveys used in the Weatherization Assistance Program Evaluation.
Provide estimates of the hour burden of the collection of information. The statement should indicate the number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated. Unless directed to do so, DOE should not conduct special surveys to obtain information on which to base hour burden estimates. Consultation with a sample fewer than 10 potential respondents is desirable.
An attachment shows the burden estimate for each element of the data collection effort as well as the total burden. Ten different surveys will be administered to representatives of various populations (states, local agencies, and utilities). In addition, a series of data requests will be made to local agencies and utility companies related to the collection of billing records, metering data, and the identification of study subjects. The above-mentioned survey instruments and household data request forms are attached to this submission for OMB review. Data will be collected from 50 states, 900 local weatherization agencies, 1,000 utility companies, for a total of 1,950 respondents. This emergency information collection request does not entail any surveys of any households that would take place during the Census embargo of such activities.
It is estimated that the hour burden for all respondents combined will total 69,136 hours over the entire study period. The data collection effort will span two years, from Spring 2010 to late 2012, but each of the individual components shown in the attachment will require less than the entire two year period. Hour burden estimates were prepared separately for each individual data collection effort based on the nature of the information requested, the instrument used, experience with similar data collection efforts in the previous National Weatherization Assistance Program Evaluation, and feedback provided by survey pre-testers.
Provide an estimate for the total annual cost burden to respondents or recordkeepers resulting from the collection of information.
No costs are anticipated for (a) capital and start-up costs or (b) an operation and maintenance and purchase of services component.
Provide estimates of annualized cost to the Federal government.
The additional cost for this data collection is $8.2 million for an independent subcontractor to Oak Ridge National Laboratory to collect the data. The average annual cost for this three-year project is $2.73 million.
Explain the reasons for any program changes or adjustments reported in Items 13 (or 14) of OMB Form 83-I.
This is a new data collection. There are no changes.
For collections whose results will be published, outline the plans for tabulation and publication.
The evaluation of the Weatherization Assistance Program will be performed starting in the Spring 2010 and will be completed in late 2012. Primary actions that will occur during the evaluation include: (a) the collection of survey information from states, agencies, agency staff, and occupants; (b) the collection of utility billing data; (c) the collection of instrumented or metered data; (d) analysis of data; and (e) preparation of reports that present the data collected and analysis results.
Preliminary information needed to select the necessary samples of states, agencies, and housing units will be collected from all 50 states (and the District of Columbia) and all ~900 agencies involved with the Program in PY’s 2007 and 2008 using two survey instruments (DF1: All States Agencies Information Data Form and DF10: All Agencies Program Overview Dataform) at the very start of the evaluation. These same states and agencies will be surveyed again shortly thereafter using two surveys (S1: All States Program Information Survey and S2: All Agencies Program Information Survey) to collect basic information that characterizes the Program and how it was implemented by states and agencies in PY 2008. At approximately the same time, a sample of 400 agencies will be surveyed to collect detailed Program implementation information (S3: Subset of Agencies Detailed Program Information Survey) and detailed information on a sample of ~20,000 housing units weatherized in PY’s 2007 and 2008 (DF2: Housing Unit Information Data Form and DF3: Building Information Data Form). Some or all of the same 400 agencies will be given a number of different household data requests for the purpose of selecting dwelling units for the billing, submetering, and instrumented measurement studies. Approximately 1,000 utilities will also be contacted with requests for billing data for the relevant households
Natural gas and electricity billing data both before and after weatherization will be collected from utilities serving the sample of ~20,000 weatherized housing units and a similar number of control housing units between Spring 2010 and the end of 2010. Instrumented measurements of indoor temperature, indoor humidity, carbon monoxide (CO), radon, asbestos, mold, pollen, and refrigerator temperature will be made in 309 housing units weatherized in PY 2010 and 59 non-weatherized control units between early 2010 and early 2011 (measurements will made both before and after weatherization). Submetered heating energy consumption data will be collected in a sample of 192 weatherized and 192 control housing units (both single-family and mobile homes) heated by fuel oil or propane, and 24 multifamily buildings heated by fuel oil, between early 2010 and early 2011. Submetered air conditioning electricity consumption data will be collected on a sample of 132 weatherized and 132 control homes (both single-family and mobile homes) between early 2010 and late 2011.
Energy billing data will be analyzed by three methods: the Princeton Scorekeeping Method (PRISM), the ORNL aggregate method, and a third method selected after review of the literature. PRISM is an established and well documented analysis method that has served as a standard analysis method for energy evaluations for many years. The ORNL aggregate method applies the same basic logic of the PRISM approach to billing and weather databases aggregated over many houses to determine an overall program effect. The advantage of the ORNL aggregate method is that few if any houses are excluded from the analysis, whereas a greater number of houses must at times be excluded from an analysis using PRISM because the data do not conform to the linear model assumed. The third method will be selected after a review of state-of-the-art techniques such as Statistically Adjusted Engineering models, Analysis of Covariance models, Conditional Demand Analysis, and fixed-effect models. Three methods are being used because of the importance of measuring the energy savings achieved by the Program as part of the evaluation, to allow comparisons of energy savings achieved under this evaluation of the Program as implemented in PY’s 2007 and 2008 with a previous evaluation of the Program based on PY 1989 data which used PRISM as the primary energy analysis method, and to establish methods that should be used in future national and state-level Program evaluations.
Characterization and survey data, submetered energy data, and other instrumented/metered data will be analyzed and/or tabulated using standard statistical techniques. For housing units in which submetered data were collected, pre- and post-weatherization energy use models will be developed by regressing weekly or daily energy consumption (the dependent variable) versus the temperature difference between the indoors and outdoors for each consumption period (the independent variable). Multivariable regressions will be used to investigate factors that impact energy savings and cost effectiveness.
After the survey information and instrumented/metered data have been collected and analyzed, a series of 9 reports will be written to present the survey information and analysis results. These reports will be published as follows:
Program characterization–A report describing the characteristics of the Program and how it is implemented will be published in December 2010.
Air and duct sealing and furnace work–A report describing the data collected by agencies on air sealing, duct sealing, furnace tune-ups, and furnace replacements and the results of analyzing these data will be published in March 2011.
Submetered heating-energy savings–Four reports documenting how each of four submetered field tests was performed (single-family homes heated by fuel oil, single-family homes heated by propane, mobile homes heated by propane, and multifamily buildings heated by fuel oil) and the heating energy savings measured under each field test will be published in March 2012.
Program energy savings–A report documenting the billing data energy analyses and the energy savings achieved in the housing units weatherized by the Program in PY 2006 will be published in March 2012.
Non-energy impacts–A report documenting the non-energy impacts that can be attributable to the Program will be published in March 2012.
Impact assessment–A report that draws together the Program characterization, energy savings, and non-energy impact studies along with additional analysis on Program cost effectiveness, explanatory factors, and attribution of energy savings to the Program will be published in September 2012.
Process assessment–A report that documents findings on how well the Program is being implemented, with a special focus on audits, client education, training, and monitoring, will be published in September 2012.
Submetered air-conditioning electricity savings–A report documenting how the air-conditioning field test was performed, the air-conditioning electricity use of Program houses, and the electricity savings measured under the field test will be published in September 2012.
Synthesis–A report that draws together all the findings presented in the previous reports and provides recommendations for Program improvement will be published in December 2012.
If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons why display would be inappropriate.
Not seeking approval not to display the expiration date for OMB. Expiration date display is not inappropriate.
Explain each exception to the certification statement identified in Item 19 of OMB Form 83-I.
No exceptions are being requested to the certification requirements.
| File Type | application/msword |
| Author | Joel F Eisenberg |
| Last Modified By | crutcev |
| File Modified | 2010-04-07 |
| File Created | 2010-04-07 |