EIA-882T OMB SS Part A 031813

SUPPORTING STATEMENT

Generic Clearance for Questionnaire Testing and Research

EIA-882T

OMB No. 1905-0186

A. JUSTIFICATION

1. Collection of Information

The U.S. Energy Information Administration (EIA) requests approval for a three-year extension of the Office of Management and Budget (OMB) to utilize qualitative and quantitative methodologies to pretest questionnaires and validate the quality of the data that is collected on the EIA survey forms. This authority would allow EIA to conduct pretest surveys, pilot surveys, respondent debriefings, cognitive interviews, usability interviews, and focus groups. Through the use of these methodologies, EIA will improve the quality of data being collected, reduce or minimize respondent burden, increase agency efficiency, and improve responsiveness to the public. This authority would also allow EIA to improve data collection in order to meet the needs of EIA’s customers while also staying current in the evolving nature of the energy industry.

The specific methods proposed for the coverage by this clearance are as follows:

Field Testing. Field testing surveys conducted under this clearance will generally be methodological studies of 500 cases or less. The samples may not be statistically representative because they will be designed to clarify particular issues rather than to be representative of the universe of survey respondents. Collection may be on the basis of convenience, e.g., limited to specific geographic locations, but the selection of sample cases will not be completely arbitrary. The sample designs will be determined at the time of development and will vary based on the content of the information collection or survey being tested.

Pilot Surveys. Pilot surveys conducted under this clearance will generally be methodological studies of 500 cases or less, but will always employ statistically representative samples. The pilot surveys will replicate all components of the methodological design, sampling procedures (where possible) and questionnaires of the full scale survey. Pilots will normally be utilized when EIA undertakes a complete redesign of a particular data collection methodology or when EIA undertakes data collection in new areas, such as greenhouse gases or alternative fueled motor vehicle transportation system studies.

Respondent Debriefings. Respondent debriefings conducted under this clearance will generally be methodological studies of 500 cases or less, involving either purposive or statistically representative samples. The debriefing form is administered after a respondent completes a questionnaire either in paper format, electronically, or through in-person interviews. The debriefings contain probing questions to determine how respondents interpret the survey questions and whether they have problems in completing the survey/questionnaire. Respondent debriefings also are useful in determining potential issues with data quality and in determining a more accurate respondent burden measure. This structured approach to debriefing enables both quantitative and qualitative analyses of data when administered to a statistically representative sample of respondents and allows EIA to improve its understanding of variance for the items in the questionnaire.

Cognitive Interviews. Cognitive interviews are typically one-on-one interviews in which the respondent is usually asked to "think aloud" or is asked “retrospective questions” as he or she answers survey questions, reads survey materials, or completes other activities as part of a survey process. A number of different techniques may be involved, including asking respondents to paraphrase questions and asking respondents probing questions to determine how they come up with their answers. The objective of these interviews is to identify problems of ambiguity or misunderstanding, to identify other difficulties respondents have answering questions, and to reduce measurement error in a survey.

Usability Interviews. Usability interviews are similar to cognitive interviews in which a respondent is typically asked to “think aloud” or asked “retrospective questions” as he or she reviews an electronic questionnaire, Web site and/or associated materials. The object of a usability interview is to make sure that electronic questionnaires, Web sites and other associated materials are user-friendly, thus allowing respondents to easily and intuitively navigate the electronic item and find the information that they seek.

Focus Groups. Focus groups involve group sessions guided by a moderator who follows a topic guide containing questions or topics focused on a particular issue rather than adhering to a standardized questionnaire. Focus groups are useful for surfacing and exploring issues with populations of interest, e.g., from a specific group of stakeholders.

Procedures for Clearance

Before testing activity is undertaken, EIA will provide OMB with a memo describing the study to be conducted and a copy of questionnaires and debriefing materials that will be used. Depending on the stage of questionnaire development, this may be a printed questionnaire, a set of prototype items showing each item type to be used and the range of topics to be covered by the questionnaire, or an interview script. When split sample experiments are conducted, either in small group sessions or as part of a field test, the different versions questionnaires to be used will be provided. For a test of alternative procedures, the description and rationale for the procedures will be submitted. A brief description of the planned field activity will also be provided. EIA requests that OMB raise comments on substantive issues within 10 working days of receipt.

2. Legal Authority

The authority for these voluntary information gathering activities is provided by the following provisions:

Section 13(b) of the Federal Energy Administration Act of 1974 (FEA Act) (P.L. 93-275) [15 U.S.C. '772(b)] states:

"All persons owning or operating facilities or business premises who are engaged in any phase of energy supply or major energy consumption shall make available to the (Secretary) such information and periodic reports, records, documents and other data, relating to the purposes of this Act, including full identification of all data and projections as to source, time, and methodology of development, as the (Secretary) may prescribe by regulation or order as necessary or appropriate for the proper exercise of functions under this Act."

The functions of the FEA Act are set forth in Section 5(b) of the Act [15 U.S.C. '764(b)], which states that the Secretary shall, to the extent he is authorized by Section 5(a) of the Act,

"(2) assess the adequacy of energy resources to meet demands in the immediate and longer range future for all sectors of the economy and for the general public; . . .

(6) assure that energy programs are designed and implemented in a fair and efficient manner so as to minimize hardship and inequity while assuring that the priority needs of the Nation are met; . . .

(9) collect, evaluate, assemble, and analyze energy information on reserves, production, demand, and related economic data;"

As the authority for invoking Section 5(b) above, Section 5(a) (15 U.S.C. '764(a)) in turn, states:

"Subject to the provisions and procedures set forth in this Act, the (Secretary) shall be responsible for such actions as are taken to assure that adequate provision is made to meet the energy needs of the Nation. To that end, he shall make such plans and direct and conduct such programs related to the production, conservation, use, control, distribution, rationing, and allocation of all forms of energy as are appropriate in connection with only those authorities or functions---

(3) Otherwise specifically vested in the (Secretary) by the Congress;"

Authority for invoking Section 5(a) of the FEA Act is provided in turn by Sections 52(a), [15 U.S.C. '790a)] which states that the [Administrator of the EIA]:

"(Shall) establish a National Energy Information System. . . (which) shall contain such information as is required to provide a description of and facilitate analysis of energy supply and consumption. . .

(b) . . . the System shall contain such energy information as is necessary to carry out the Administration's statistical and forecasting activities, and . . . such energy information as is required to define and permit analysis of . . .

(1) the institutional structure of the energy supply system including patterns of ownership and control of mineral fuel and nonmineral energy resources and the production, distribution, and marketing of mineral fuels and electricity;

(2) the consumption of mineral fuels, nonmineral energy resources, and electricity by such classes, sectors, and regions as may be appropriate for the purposes of this Act;

. . . (4) the comparability of energy information and statistics that are supplied by different sources; . . .

3. The Purpose of Data Collection

The information gathered will be used to help modify and develop better questionnaires and data collection procedures. The end result will be improvements in the quality of data being collected by EIA, reduction or minimization of respondent burden, increased agency efficiency, and improved responsiveness to the public.

Results of these efforts will also provide EIA a better understanding of the energy areas being studied and how respondents comprehend survey questions. Methodological results will be shared with other U.S. Department of Energy (DOE) components and other agencies in seminars and through presentations at meetings, e.g., at the American Statistical Association's meetings and the Federal Committee on Statistical Methodology Research conference. No substantive results of the questions will be shared.

Annual reports of the results of generic questionnaire testing, evaluation, and research are provided to OMB. During the past three years, EIA has used this clearance for a variety of studies including:

• Residential Energy Consumption Survey (RECS): The evaluation of RECS assessed how well respondents comprehend and the accuracy of responses to the questions used in 2005, identified potentially sensitive questions that should be avoided, tested the accuracy of reworded or new 2009 questions, and identified a questionnaire design and order to avoid inconsistent responses.

• Form EIA-857 “Monthly Report of Natural Gas Purchases and Deliveries to Consumers: In two separate studies (2009, 2010), EIA investigated how companies record and report monthly natural gas deliveries and possible advances in the collection form that would result in more accurate information without increasing burden to the respondent.

• Petroleum Supply Survey Forms (2010): EIA investigated the time spent by companies collecting and collating data, completing the form, submitting the survey to EIA, and doing follow-up with EIA.

• Web usability testing (2010): EIA used an online testing tool called “Loop 11” to better understand the usability of EIA’s website.

• Form EIA-815 “Monthly Bulk Terminal and Blenders Report” (2010): To improve reporting accuracy and estimated variance, EIA studied the definition of bulk terminals, storage classification at bulk terminals, reporting of trans-shipment tanks, and reclassification of fuels or new fuels and related topic of fuel blending.

• Form EIA-782C “Monthly Report of Prime Supplier Sales of Petroleum Products Sold for Local Consumption”: To prepared for the 2013 Petroleum Marking and Supply clearance packages, EIA investigated the following terms: Fungibility, blends, reformulated and conventional gasoline, and the EIA-782C Exclusionary list.

4. Use of Improved Technology

When a pretesting methodology employs automated methods for its data collection, the research conducted under this submission will also utilize automated data collection techniques.

5. Efforts to Identify Duplication

These tasks will not duplicate any other survey/questionnaire design or pretest work being done by EIA or other Federal agencies. One of the purposes of this request is to stimulate additional research that would not otherwise be performed due to time and cost constraints. This research may involve collaboration with other agencies, especially the Office of Management and Budget, Bureau of Labor Statistics, Census, and the National Science Foundation.

EIA will use existing information to the maximum extent possible by reviewing results of earlier evaluations of survey data before attempting to revise existing questionnaires or design new ones. EIA survey staff also will keep up-to-date on private and public survey research and results.

6. Collection of Information Involving Small Establishments

One purpose of these efforts is to develop survey methodologies and questionnaires to which responding is easier for small establishments and individuals, thus minimizing burden. These methods should also give small establishments a vehicle for input, with a minimum amount of burden, on the use of improved information technology (electronic data maintenance, submission, or disclosure) to substitute for paper survey forms.

7. Consequences of Less Frequent Data Collection

Because this clearance is an on-going effort, it has the potential to have an immediate impact on all of EIA’s data collections. If this project were not carried out, the quality of the data collected in the surveys would suffer.

8. Special Circumstances

All the guidelines listed in the OMB guidelines are met. There are no special circumstances.

9. Federal Register and Consultation Outside of EIA

Federal Register. A Federal Register notice was published in Volume 77 FR 15092 on 03/14/2012. This notice did not generate public comment.

Outside Consultation. Consultation with staff from other Federal agencies that sponsor surveys conducted by EIA will occur in conjunction with the testing program for each survey. Consultation with staff from other Federal testing facilities may also occur as part of joint research efforts. These consultations will include discussions concerning potential response problems, clarity of questions and instructions, and other aspects of respondent burden. Additional efforts to consult with potential respondents to obtain their views on the availability of data, clarity of instructions, etc. may be undertaken as part of the testing conducted under this clearance.

10. Payment or Gift to Respondents

Respondents for activities conducted in the laboratory (e.g. cognitive interviews and focus groups) under this clearance may receive compensation for travel and participation. This practice has proven necessary and effective in recruiting subjects to participate in such research, and is also employed by the other Federal cognitive laboratories. Research on incentives that may be conducted under this clearance may also involve nonmonetary incentives. The Office of Management and Budget has noted that effectiveness of such incentives is a worthwhile research topic. If incentives need to be proposed for any research activity under this clearance, justification will be provided and we will work closely with OMB on the incentive strategy to be employed. EIA will typically propose incentives at the level approved by the Office of Management and Budget for cognitive laboratories and focus groups (currently up to $40 for cognitive interviews and up to $75 for focus groups). If a higher level incentive is proposed for approval, a meaningful justification will be provided.

11. Confidentiality and Privacy Concerns

The data collected from respondents will be tabulated and analyzed only for the purpose of evaluating the research in question. All respondents who participate in research under this clearance will be informed that their participation is voluntary. Survey results will only be presented internally to EIA stakeholders and will be presented in de-identified form.

For studies and pretests where proprietary data is obtained, respondents will be advised, in writing, that their participation is voluntary and the following confidentiality provision will be made:

The information reported on the form will be protected and not disclosed to the public to the extent that it satisfies the criteria for exemption under the Freedom of Information Act (FOIA), 5 U.S.C. §552, the Department of Energy (DOE) regulations, 10 C.F.R. §1004.11, implementing the FOIA, and the Trade Secrets Act, 18 U.S.C. §1905.

12. Questions of a Sensitive Nature

Most of the questions being asked will not be of a sensitive nature and should not pose a problem to respondents. However, it is possible that in developing surveys potentially sensitive questions may be included. One of the purposes of these efforts is to identify such questions, determine sources of sensitivity, and alleviate them insofar as possible before an actual survey is conducted.

13. Burden of the Collection of Information

The total estimated respondent burden is 3000 hours for the period of March 2013 through February 2016. These hours will be distributed as follows:

Respondent Burden Hours

Mar 2013 – Feb 2014 1000 hours

Mar 2014 – Feb 2015 1000 hours

Mar 2015 – Feb 2016 1000 hours

Total 3000 hours

This estimate is based on our experience with past clearances and our anticipation that the research will require approximately one hour per respondent.

A variety of forms will be used in conducting the research under this clearance, and the exact number of different forms, length of each form, and number of subject/ respondents per form are unknown at this time.

14. Estimate of Total Annual Cost

Additional capital, start-up, or operation and maintenance costs for respondents should not be necessary for any data collections under this clearance. There is typically no cost to respondents for participating in the research being conducted under this clearance, except for their time to complete the questionnaire.

15. Annualized Cost to the Federal Government

Additional cost to the Federal Government is not anticipated. Expenses (equipment, overhead, printing, and support staff) will be incurred by EIA components as part of their normal operating budgets. We will include information about costs in the individual submissions to OMB.

16. Program Changes or Adjustments

In a previous clearance, EIA requested 1,000 hours across all three years. In this current clearance EIA is requesting 1,000 hours for each year for a total of 3,000 hour across the three years. This increase is due to our anticipation of conducting further research to improve the quality of data being collected, reduce or minimize respondent burden, increase agency efficiency, and improve responsiveness to the public.

17. Schedule of Data Collection and Publication

The information will be used for questionnaire development and to improve the quality of EIA’s data. The results will not be published in EIA publications. Some papers referencing the pretesting may be written and presented at public meetings or published in EIA service reports. Due to the nature of this clearance, there is no time schedule for publication of this information. Work will continue throughout the duration of the three-year clearance.

18. Request to Not Display Expiration Date

EIA will display the expiration date for OMB approval of the information collection. No exemption is requested.

19. Exceptions to the Certification Statement

There are no exceptions to the certification.

Page 7 of 7