EIA-882T OMB SS Part A 012216

SUPPORTING STATEMENT

PART A

JUSTIFICATION

Generic Clearance for Questionnaire Testing and Research

EIA-882T

OMB No. 1905-0186

A. JUSTIFICATION

1. Collection of Information

The U.S. Energy Information Administration (EIA) requests approval for a three-year extension of the Office of Management and Budget (OMB) to utilize qualitative and quantitative methodologies to pretest questionnaires and validate the quality of the data that is collected on the EIA survey forms. This authority would allow EIA to conduct pretest surveys, pilot surveys, respondent debriefings, cognitive interviews, usability interviews, and focus groups. Through the use of these methodologies, EIA will improve the quality of data being collected, reduce or minimize respondent burden, increase agency efficiency, and improve responsiveness to the public. This authority would also allow EIA to improve data collection in order to meet the needs of EIA’s customers while also staying current in the evolving nature of the energy industry.

The specific methods proposed for the coverage by this clearance are as follows:

Field Testing. Field testing surveys conducted under this clearance will generally be methodological studies. The samples may not be statistically representative because they will be designed to clarify particular issues rather than to be representative of the universe of survey respondents. Collection may be on the basis of fuel market coverage related to the issues that are the subject to the research, market size of the respondent, position of the respondent in the upstream and/or downstream market for a particular fuel group, and convenience, e.g., limited to specific geographic locations. The sample sizes and designs will be determined at the time of development and will vary based on the content of the information collection or survey being tested.

Pilot Surveys. Pilot surveys conducted under this clearance will generally be methodological studies, but will always employ statistically representative samples. The pilot surveys will replicate all components of the methodological design, sampling procedures (where possible) and questionnaires of the full scale survey. Pilots will normally be utilized when EIA undertakes a complete redesign of a particular data collection methodology or when EIA undertakes data collection in new areas, such as greenhouse gases or alternative fueled motor vehicle transportation system studies.

Respondent Debriefings. Respondent debriefings conducted under this clearance will generally be methodological or cognitive research studies. The debriefing form is administered after a respondent completes a questionnaire either in paper format, electronically, or through in-person interviews. The debriefings contain probing questions to determine how respondents interpret the survey questions, how much time and effort was spent completing the questionnaire, and whether they have problems in completing the survey/questionnaire. Respondent debriefings also are useful in determining potential issues with data quality and in estimating respondent burden.

Cognitive Interviews. Cognitive interviews are typically one-on-one interviews in which the respondent is usually asked to "think aloud" or is asked “retrospective questions” as he or she answers survey questions, reads survey materials, or completes other activities as part of a survey process. A number of different techniques may be involved, including asking respondents what specific words or phrases mean, and asking respondents probing questions to determine how they estimate or calculate specific data elements on a survey form. The objective of these interviews is to identify problems of ambiguity or misunderstanding, to identify other difficulties respondents have answering questions, and to reduce measurement error from estimates based on a survey.

Usability Interviews. Usability interviews are similar to cognitive interviews in which a respondent is typically asked to “think aloud” or asked “retrospective questions” as he or she reviews an electronic questionnaire, website and/or associated materials, or hard copy form. The object of a usability interview is to make sure that respondents can easily and intuitively navigate electronic questionnaires, websites and other associated materials, or other survey instruments. .

Focus Groups. Focus groups involve group sessions guided by a moderator who follows a topic guide containing questions or topics focused on a particular issue rather than adhering to a standardized questionnaire. Focus groups are useful for surfacing and exploring issues with populations of interest, e.g., from a specific group of stakeholders.

Procedures for Clearance

Before testing activity is undertaken, EIA will provide OMB with a memo describing the study to be conducted and a copy of questionnaires and debriefing materials that will be used. Depending on the stage of questionnaire development, this may be a printed questionnaire, a set of prototype items showing each item type to be used and the range of topics to be covered by the questionnaire, or an interview script. When split sample experiments are conducted, either in small group sessions or as part of a field test, the different versions questionnaires to be used will be provided. For a test of alternative procedures, the description and rationale for the procedures will be submitted. A brief description of the planned field activity will also be provided. EIA requests that OMB raise comments on substantive issues within 10 working days of receipt.

2. Legal Authority

The authority for these voluntary information gathering activities is provided by the following provisions:

Section 13(b) of the Federal Energy Administration Act of 1974 (FEA Act) (P.L. 93-275) [15 U.S.C. '772(b)] states:

"All persons owning or operating facilities or business premises who are engaged in any phase of energy supply or major energy consumption shall make available to the (Secretary) such information and periodic reports, records, documents and other data, relating to the purposes of this Act, including full identification of all data and projections as to source, time, and methodology of development, as the (Secretary) may prescribe by regulation or order as necessary or appropriate for the proper exercise of functions under this Act."

The functions of the FEA Act are set forth in Section 5(b) of the Act [15 U.S.C. '764(b)], which states that the Secretary shall, to the extent he is authorized by Section 5(a) of the Act,

"(2) assess the adequacy of energy resources to meet demands in the immediate and longer range future for all sectors of the economy and for the general public; . . .

(6) assure that energy programs are designed and implemented in a fair and efficient manner so as to minimize hardship and inequity while assuring that the priority needs of the Nation are met; . . .

(9) collect, evaluate, assemble, and analyze energy information on reserves, production, demand, and related economic data;"

As the authority for invoking Section 5(b) above, Section 5(a) (15 U.S.C. '764(a)) in turn, states:

"Subject to the provisions and procedures set forth in this Act, the (Secretary) shall be responsible for such actions as are taken to assure that adequate provision is made to meet the energy needs of the Nation. To that end, he shall make such plans and direct and conduct such programs related to the production, conservation, use, control, distribution, rationing, and allocation of all forms of energy as are appropriate in connection with only those authorities or functions---

(3) Otherwise specifically vested in the (Secretary) by the Congress;"

Authority for invoking Section 5(a) of the FEA Act is provided in turn by Sections 52(a), [15 U.S.C. '790a)] which states that the [Administrator of the EIA]:

"(Shall) establish a National Energy Information System. . . (which) shall contain such information as is required to provide a description of and facilitate analysis of energy supply and consumption. . .

(b) . . . the System shall contain such energy information as is necessary to carry out the Administration's statistical and forecasting activities, and . . . such energy information as is required to define and permit analysis of . . .

(1) the institutional structure of the energy supply system including patterns of ownership and control of mineral fuel and nonmineral energy resources and the production, distribution, and marketing of mineral fuels and electricity;

(2) the consumption of mineral fuels, nonmineral energy resources, and electricity by such classes, sectors, and regions as may be appropriate for the purposes of this Act;

. . . (4) the comparability of energy information and statistics that are supplied by different sources; . . .

3. The Purpose of Data Collection

The information gathered will be used to help modify and develop better questionnaires and data collection procedures. The end result will be improvements in the quality of data being collected by EIA, reduction or minimization of respondent burden, increased agency efficiency, and improved responsiveness to the public.

Results of these efforts will also provide EIA a better understanding of the energy areas being studied and how respondents comprehend survey questions. Methodological results will be shared with other U.S. Department of Energy (DOE) components and other agencies in seminars and through presentations at meetings, e.g., at the American Statistical Association's meetings and the Federal Committee on Statistical Methodology Research conference.

During the past three years, EIA has used this clearance for a variety of studies including:

• EIA-915, Monthly Gas Processing and Liquids Report

This research examined the response process as a whole, the ability and level of effort by potential respondents to complete a newly designed survey instrument. Respondents currently answer similar questions on the EIA-64A and EIA-816 that could be consolidated into Form EIA-915. In an effort to understand the response process, the study investigated possible barriers to reporting requested information such as data availability and timing of the survey request.

• EIA-857: Monthly Report of Natural Gas Purchases and Deliveries to Consumers

EIA conducted this study to gain a better understanding of how the current respondent completes the survey and to determine the level of effort required to respond to the data request. EIA sought to better understand how natural gas companies are reporting the total costs and the associated volumes of natural gas being delivered, the process undertaken by the respondent in providing costs and the barriers that exist to completing the form.

• EIA-914 (Expansion): Natural Gas Production Report

In 2015 EIA-914 survey, EIA added the collection of crude oil and lease condensate production from well operators. During 2014, EIA did cognitive research to explore collecting this information both as an aggregate and separated by specific API gravities and sulfur content by state. This research was conducted as part of an effort to understand the potential measurement error in this proposed expansion. Cognitive interviews were conducted with existing EIA-914 respondents that focused on assessing the availability of the data in respondents’ records and to understand the cognitive burden required from the respondents to access and tabulate the information in their records to report domestic crude oil production.

• Residential Energy Consumption Survey (RECS)

To prepare for the next RECS, EIA updated the household characteristics questions within this survey. Changes included adding questions to collect information on new technologies and revising questions and response options relating to appliance usage to improve respondent clarity and data quality. EIA conducted in-person cognitive interviews and online question pretesting with respondents from the general population and panelists of online testing platforms. The primary focus of this research was to improve data quality by pretesting new questions and evaluating improvements to usage questions:

• Residential Energy Consumption Survey (RECS Cities Pilot)

This research was conducted in an effort to improve the Household Survey phase of EIA’s RECS program (Form 457A). EIA tested the use of collecting the information as a self-administered web survey or mailing it as a hardcopy form to be completed instead of collecting the information by a personal interview. In spring 2015, EIA conducted a small pilot test of web and mail modes of collection in five U.S. cities (the “Cities Pilot”). This research included the use of open-ended responses to specific appliance usage questions.

• Business Knowledge and Perceptions of EA Efforts to Protect Confidentiality

This research studied the perceptions the survey respondents have regarding EIA efforts to protect the confidentiality of information reported on survey forms, their understanding of the confidentiality pledges used on EIA surveys, and respondent perceptions of researchers accessing confidential data for statistical research purposes.

• Residential Energy Consumption Survey (RECS National Pilot)

This study involved the Household Survey phase of EIA’s RECS program (Form 457A). The National Pilot test of the Household Survey is a follow up study from the Cities Pilot test to assess the quality and effectiveness of using web and mail data collection modes. EIA tested the feasibility, cost effectiveness, time efficiency, and response validity of conducting the RECS Household Survey in self-administered (web and mail) modes. This pilot test incorporated key design elements and lessons learned from the Cities Pilot in order to compare these results against our benchmark RECS computer-assisted personal interviewing (CAPI) collection. 

4. Use of Improved Technology

When a pretesting methodology employs automated methods for its data collection, the research conducted under this submission will also utilize automated data collection techniques. EIA seeks to explore the use of web cameras to conduct cognitive research personal interviews with the respondent and interviewer observing each other from a monitor screen. This interviewing mode will help reduce costs while preserving the benefits of an in-person interview.

5. Efforts to Identify Duplication

These tasks will not duplicate any other survey/questionnaire design or pretest work being done by EIA or other Federal agencies. One of the purposes of this request is to stimulate additional research that would not otherwise be performed due to time and cost constraints. This research may involve collaboration with other agencies, especially the Office of Management and Budget, Bureau of Labor Statistics, Census, and the National Science Foundation.

EIA will use existing information to the maximum extent possible by reviewing results of earlier evaluations of survey data before attempting to revise existing questionnaires or design new ones. EIA survey staff also will keep up-to-date on private and public survey research and results.

6. Collection of Information Involving Small Establishments

One purpose of these efforts is to develop survey methodologies and questionnaires to which responding is easier for small establishments and individuals, thus minimizing burden. These methods should also give small establishments a vehicle for input, with a minimum amount of burden, on the use of improved information technology (electronic data maintenance, submission, or disclosure) to substitute for paper survey forms.

7. Consequences of Less Frequent Data Collection

Because this clearance is an on-going effort, it has the potential to have an immediate impact on all of EIA’s data collections. If this project were not carried out, the quality of the data collected in the surveys would suffer.

8. Special Circumstances

All the guidelines listed in the OMB guidelines are met. There are no special circumstances.

9. Federal Register and Consultation Outside of EIA

Federal Register. A Federal Register notice was published on October 30, 2015 (FR Vol. 80, No. 210, p. 66894). There were no comments to the Federal Register Notice.

Outside Consultation. Consultation with staff from other Federal agencies that sponsor surveys conducted by EIA will occur in conjunction with the testing program for each survey. Consultation with staff from other Federal testing facilities may also occur as part of joint research efforts. These consultations will include discussions concerning potential response problems, clarity of questions and instructions, and other aspects of respondent burden. Additional efforts to consult with potential respondents to obtain their views on the availability of data, clarity of instructions, etc. may be undertaken as part of the testing conducted under this clearance.

10. Payment or Gift to Respondents

Respondents for activities conducted in the laboratory (e.g. cognitive interviews and focus groups) under this clearance may receive compensation for travel and participation. This practice has proven necessary and effective in recruiting subjects to participate in such research, and is also employed by the other Federal cognitive laboratories. Research on incentives that may be conducted under this clearance may also involve nonmonetary incentives. The Office of Management and Budget has noted that effectiveness of such incentives is a worthwhile research topic. If incentives need to be proposed for any research activity under this clearance, justification will be provided and we will work closely with OMB on the incentive strategy to be employed. EIA’s usual incentive payments have ranged from $10 - $20 and continue to propose incentives at the level approved by the Office of Management and Budget for cognitive laboratories and focus groups. If a higher level incentive is proposed for approval, a meaningful justification will be provided.

11. Confidentiality and Privacy Concerns

The data collected from respondents will be tabulated and analyzed only for the purpose of evaluating the research in question. Information from cognitive interviews is de-identified with all direct identifying information removed and reference to the respondent is written in the third person singular and plural tenses. All respondents who participate in research under this clearance will be informed that their participation is voluntary. Survey results will only be presented internally to EIA stakeholders and will be presented in de-identified form.

For studies and pretests where proprietary data are obtained, respondents will be advised, in writing, that their participation is voluntary and the following confidentiality provision will be made:

The information reported on the form will be protected and not disclosed to the public to the extent that it satisfies the criteria for exemption under the Freedom of Information Act (FOIA), 5 U.S.C. §552, the Department of Energy (DOE) regulations, 10 C.F.R. §1004.11, implementing the FOIA, and the Trade Secrets Act, 18 U.S.C. §1905.

12. Questions of a Sensitive Nature

Most of the questions being asked will not be of a sensitive nature and should not pose a problem to respondents. However, it is possible that in developing surveys potentially sensitive questions may be included. One of the purposes of these efforts is to identify such questions, determine sources of sensitivity, and alleviate them insofar as possible before an actual survey is conducted.

13. Burden of the Collection of Information

The total estimated respondent burden is 6,000 hours for the period of March 2016 through February 2019. These hours will be distributed as follows:

Mar 2016 – Feb 2017: 2,000 hours

Mar 2017 – Feb 2018: 2,000 hours

Mar 2018 – Feb 2019: 2,000 hours

Total: 6,000 hours

As EIA works to better meet its organizational goals, we anticipate more collaboration with respect to enhanced survey/product development planning. EIA plans to expand cognitive research activity for evaluating proposals to modify surveys and to more accurately estimate reporting burden. This estimate is based on recent trends to increase the electronic data collection and processing of survey data, our experience with past clearances, and our anticipation that the research will require approximately one hour per respondent.

A variety of forms will be used in conducting the research under this clearance, and the exact number of different forms, length of each form, and number of subject/ respondents per form are unknown at this time.

14. Estimate of Total Annual Cost

Additional capital, start-up, or operation and maintenance costs for respondents should not be necessary for any data collections under this clearance. There is typically no cost to respondents for participating in the research being conducted under this clearance, except for their time to complete the questionnaire.

15. Annualized Cost to the Federal Government

Additional cost to the Federal Government is not anticipated. Expenses (equipment, overhead, printing, and support staff) will be incurred by EIA components as part of their normal operating budgets. We will include information about costs in the individual submissions to OMB.

16. Program Changes or Adjustments

In a previous clearance, EIA requested 9,018 hours across all three years. In this current clearance EIA is requesting 2,000 hours for each year for a total of 6,000 hours across the three years. This decrease is due to the one-time research efforts in 2015 that were associated with the Cities Pilot test and National Pilot test for using alternative data collection modes for the RECS survey.

17. Schedule of Data Collection and Publication

The information will be used for questionnaire development, improve estimates of reporting costs and burden, and to improve the quality of EIA’s data. The results will not be published in EIA publications. Some papers referencing the pretesting may be written and presented at public meetings or published in EIA service reports. Due to the nature of this clearance, there is no time schedule for publication of this information. Work will continue throughout the duration of the three-year clearance.

18. Request to Not Display Expiration Date

EIA will display the expiration date for OMB approval of the information collection. No exemption is requested.

19. Exceptions to the Certification Statement

There are no exceptions to the certification.

Page 8 of 8