Findings and Recommendations from Response Burden Testing for the EIA-23L

Findings and Recommendations from
Response Burden Testing for the EIA-23L
Prepared for:
Steven Grape, Office of Oil, Gas, and Coal Supply Statistics (OGCSS)
Barbara Mariner-Volpe, OGCSS
James Kendell, OGCSS

Prepared by:
Marlana Anderson, Office of Survey Development and Statistical Integration (SDSI)
Brian Hewitt, SDSI
Jacob Bournazian, SDSI
Nanda Srinivasan, SDSI

U.S. Energy Information Administration
U.S. Department of Energy

November 12, 2015

Table of Contents
Research Objectives ...................................................................................................................................... 3
Research Methodology ................................................................................................................................. 3
Executive Summary....................................................................................................................................... 3
Question Specific Findings: ........................................................................................................................... 4
Part B Background Information ................................................................................................................ 4
Part B Response Burden Current Form ..................................................................................................... 5
Part B Response Burden County Level Form ............................................................................................ 6
Part C WRAP UP ........................................................................................................................................ 8
Appendix A .................................................................................................................................................. 10
Part A – Introduction................................................................................................................................... 10
Part B – Background Information ............................................................................................................... 10
Appendix B .................................................................................................................................................. 13

Findings from Response Burden Interviews for the EIA-23L
November 12, 2015
Page 2 of 15

Research Objectives
The Office of Oil, Gas, and Coal Supply Statistics (OGCSS) seeks to improve the accuracy of the response
burden figure on the EIA-23L survey form and explore the potential change in response burden due to
new reporting requirements for respondents. The EIA-23L, “The Annual Survey of Domestic Oil and Gas
Reserves“ collects data on proved reserves and production of crude oil, natural gas, lease condensate,
and other related products.
Specific objective of the interviews on Form EIA-23L were:



Collect information on the response burden associated with the current version of the survey
form.
Explore proposed changes to the survey form and the impact on response burden these changes
would have.

Research Methodology
SDT conducted nine exploratory interviews with current EIA-23L respondents from October 25-30, 2015.
Companies that participated in the interviews were selected based on size of their operation. Three
companies were selected from three categories: large size companies defined as annual production
greater than 1.5 million barrels of crude oil, or 15 billion cubic feet of natural gas; medium size
companies defined as annual production between 400 thousand barrels of oil, or 2 billion cubic feet of
natural gas, and 1.5 million barrels of crude oil, or 15 billion cubic feet of natural gas; and small size
companies defined as annual production less than 400 thousand barrels of crude oil ,or 2 billion cubic
feet of natural gas. The interviews lasted approximately 30 minutes. Appendix A contains the protocol
used to conduct the interviews.

Executive Summary
Overall the interviews conducted revealed a significant response burden associated with the EIA-23L
survey. The answers we received regarding the response burden varied significantly. This finding was
consistent between all three size groupings of companies. The interviews revealed that much of the
response burden can be attributed to the Reserve Information Gathering System (RIGS) online tool used
by respondents to fill out the survey. Respondents also revealed that asking them to report data at the
county level, instead of field level, would decrease the response burden associated with the EIA-23L
survey.

Findings from Response Burden Interviews for the EIA-23L
November 12, 2015
Page 3 of 15

Question Specific Findings:
Part B Background Information
FINDINGS LARGE COMPANIES
The length of time that the larger companies have been in operation ranged from 16 years to over 50
years. Although their job titles differed, the respondents for these large companies had similar job
responsibilities. The respondents’ responsibilities included coordinating and gathering data for
government reports and corporate analysis. All of the respondents within this group have filled out the
EIA-23L survey previously.

FINDINGS MEDIUM COMPANIES
The length of time that the medium companies have been in operation ranged from 15 years to over 60
years. Similar to the large companies interviewed, the job titles of the respondents for the medium
sized companies varied, however their primary job responsibilities were centered on the reporting of
government forms and the managing of data regarding oil and gas reserves. Most (2) of the medium
sized company respondents have been filling out the EIA-23L survey for 8 years. One respondent from
this group reported that they have filled out the survey for over 30+ years.

FINDINGS SMALL COMPANIES
The length of time that the small companies have been in operation ranged from 7 years to 34 years.
Similar to the medium and large sized companies interviewed, the respondents’ titles for the small
establishments differed, but the job responsibilities were comparable. The responsibilities of the
respondents for small companies were centered on data gathering and reporting. The respondents also
had varying lengths of experience in filling out the EIA-23L. The respondent with the most experience
with the survey had been filling out for 13 years, while the respondent with the least experience had
only filled out the survey once.

DISCUSSION
The companies that participated in these interviews varied by the size of their operation and by how
long the company has been operating in the oil and gas business. The respondents interviewed also had
differing job titles and differing levels of experience when filling out the EIA-23L survey. This assortment
of companies and respondents provide a range of estimates regarding the reporting burden for filing
Form EIA-23L.
Findings from Response Burden Interviews for the EIA-23L
November 12, 2015
Page 4 of 15

Part B Response Burden Current Form
FINDINGS LARGE COMPANIES
Respondents’ answers for the time it takes to gather the data necessary for the EIA-23L ranged from 1060 hours. The average for the responses was 39.3 hours.
Once the information was compiled, the respondents’ stated that the amount of time it takes to fill-in
and file the EIA-23L survey form ranged from 2-112 hours. The average for these responses was 44.7
hours.
Total reporting burden for large size companies ranged from 12-160 hours with an average reporting
burden of 84 hours.
All of the respondents reported that they completed the survey form without the assistance of others.
Two of the three respondents reported that they filled out the survey using a software program.
Another commented about how tedious it was to find the well location using the RIGS online
application.

FINDINGS MEDIUM COMPANIES
Respondents reported that the amount of time it takes them to gather the data in order to report on the
EIA-23L survey ranged from 3-80 hours. The average for the responses was 34 hours.
Once the information was compiled, the Respondents’ stated that the amount of time it takes to fill-in
and file the EIA-23L survey form ranged from 1-117 hours. The average for the responses was 66 hours.
Total reporting burden for medium size companies ranged from 20-160 hours with an average reporting
burden of 100 hours.
One of the three respondents stated that selecting the fields using the RIGS online tool takes time and
changing data is difficult. He described the RIGS application as “archaic.”

FINDINGS SMALL COMPANIES
The answers for the amount of time it takes respondents to gather the data necessary to report on the
EIA-23L ranged from 3-40 hours. The average for these responses was 16.7 hours.
The answers for the amount of time it takes small companies to fill-in and file the EIA-23L survey ranged
from 1-120 hours. The average for these responses was 40.7 hours.
Findings from Response Burden Interviews for the EIA-23L
November 12, 2015
Page 5 of 15

Total reporting burden for small size companies ranged from 4-160 hours with an average reporting
burden of 57.3 hours
DISCUSSION
The reported current response burden for companies filling out the EIA-23L form varied significantly.
The wide range of responses pertaining to the amount of hours required to complete the survey was not
related to the size of the company interviewed, nor was it connected to the respondents’ experience
with the survey. This finding was also present when respondents were asked to estimate the amount of
time it takes to collect data for the EIA-23L survey.
Two of the respondents interviewed reported that the online tool was a reason why the survey required
a significant amount of time to fill out. One of these respondents reported difficulty selecting specific
wells using the RIGS online tool.
Another one of the respondents interviewed informed us that there was a three year lag before EIA had
updated field name information in the Field Code Master List. This respondent said that changing the
incorrect field data was difficult and time consuming.
Respondents who reported a large response burden were both respondents who filled out the survey
manually and respondents who used a program to fill out the RIGS online tool.

Part B Response Burden County Level Form
FINDINGS LARGE COMPANIES
Responses by respondents for how long they anticipate it will take to collect data at the county level
EIA-23L survey form ranged from 10-24 hours. The average of these responses was 19.3 hours, a
decrease of 50.8% compared to the current survey.
Respondents’ answers for the amount of time they anticipate it will take to fill-in and file the county
level EIA-23L survey form ranged from 1-96 hours. The average for these responses was 37.7 hours, a
decrease of 15.7% compared to the current survey.
The anticipated total response burden for the county level survey for large companies ranged between
11-120 hours. The average of these responses was 57 hours, a decrease of 32.1% compared to the
current survey.
None of the respondents reported that they will need to purchase new software or equipment to
complete the new survey. Although, one of the three respondents did note that the program they used
to fill out the form online will need to be modified.
All of the respondents interviewed thought that the change to county level collection would make the
survey easier to fill out.
Findings from Response Burden Interviews for the EIA-23L
November 12, 2015
Page 6 of 15

FINDINGS MEDIUM COMPANIES
Responses by respondents for how long they anticipate it will take to collect data for the county level
EIA-23L survey form ranged from 3-80 hours. The average of these responses was 30.7 hours, a decrease
of 9.8% compared to the current survey.
Respondents’ answers for the amount of time they anticipate it will take to fill-in and file the county
level EIA-23L survey form ranged from 1-117 hours. The average for these responses was 52.7 hours, a
decrease of 20.2% compared to the current survey.
The anticipated total response burden for the county level survey for medium companies ranged
between 10-120 hours. The average of these responses was 83.3 hours, a decrease of 16.7% compared
to the current survey.
One of the three respondents for medium companies was not able to estimate how much time it would
take to gather the data. This respondent thought that the move to county level would be more difficult
because the system they used to track data was based on field level and not county level.
Two of the respondents interviewed thought that the change to county level collection would make the
survey easier to fill out.

FINDINGS SMALL COMPANIES
One of the respondents for the small companies anticipates the new survey will require 3 hours to
gather data for the count level EIA-23L survey. This was a decrease of 57.1% from their answer for the
current survey. Additionally, this respondent did not anticipate a change in the amount of hours (1) it
will take to fill-in and file the county level EIA-23L survey. The anticipated total response burden by this
respondent for the county level EIA-23L survey was 4 hours, a decrease of 50% compared to their
response for the current survey.
One of the other two respondents for small companies was not able to quantify the amount of time it
would take to complete the new survey, or the amount of time it would take to gather data for the new
survey. However, they stated that the amount of time required to complete a county level EIA-23L form
would be “greatly reduced” compared to the field level form.
The other respondent, who was not able to quantify the amount of time it would take to complete the
new survey, stated that initially the survey would be more difficult, but eventually it would be “ok.”
Two of the three respondents interviewed thought that the change to county level collection would
make the survey easier to fill out.
Findings from Response Burden Interviews for the EIA-23L
November 12, 2015
Page 7 of 15

DISCUSSION
Seven of the nine of the respondents interviewed thought that collecting data at the county level would
make the survey easier to fill out. These seven respondents also reported that the reporting level from
field level to county level would reduce the time it takes to the complete the survey.
Regarding the gathering of data for the new report, five out of nine respondents anticipated the new
survey would require less time to gather the data. Two respondents estimated the time it takes to
collect the data would remain the same. Two respondents indicated it would increase their reporting
burden.
Of the two respondents who stated the change to county level would make the survey more difficult,
one stated their current data is collected at the field level and to report county level data would make
reporting more difficult. The other of the two respondents reported that the move to reporting data at
the county level would make the reporting more difficult initially, but said it would be fine eventually.
This respondent did not elaborate why it would be more difficult initially. It should be noted that this
respondent had the least amount of experience with the EIA-23L and only filled it out once before.
Although none of the respondents reported that they would need to purchase new software or
equipment to complete the new survey. One of the respondents who use automated programs to fill
out the survey noted that the program used would have to be modified.
A majority of respondents believe the move to county level will make the survey easier and will take less
time to do. Those who did not think the move would make it easier were either inexperienced in filling
out the survey or they had a data base system based on field level data collection that may be difficult to
modify to aggregate the data at county level.

Part C WRAP UP
FINDINGS LARGE COMPANIES
Most (2) of the respondents for the large companies filled out other EIA forms. Similarly, most (2) also
commented on the difficulty of the online tool used to fill out the form. One described the RIGS online
application as “archaic” and the other described it as “tedious” to use. However, one respondent did
describe the survey as “straight forward.”
One of the respondents suggested the idea of allowing respondents to access last year’s data. Another
respondent commented that their automated program for filling out the survey no longer worked, but
could not determine if that was an issue on their end or for EIA.

Findings from Response Burden Interviews for the EIA-23L
November 12, 2015
Page 8 of 15

FINDINGS MEDIUM COMPANIES
Only one of the three respondents for medium sized companies filled out other EIA survey forms. When
asked to describe the EIA-23L, one said “it was not a terrible form” and compared it to filling out tax
forms. Another respondent stated the survey was cumbersome and cited the RIGS online tool as difficult
to use and time consuming.
One of three medium company respondents had a suggestion for improving the survey. This respondent
stated the RIGS application had programming issues and elaborated that while filling out the survey
there were times when the information entered would be deleted by the program. The other two
respondents did not have any suggestions, but one respondent did say that changing the reporting level
from field level move to county level was “a step in the right direction.”
FINDINGS SMALL COMPANIES
When asked for what they think of the survey overall respondents’ answers varied. One respondent
described the survey as time consuming. Another respondent describe it as “Not the top priority, but
not bad.” The other respondent said they “did not hate” the EIA-23L survey.
When asked for suggestion on how we can improve the survey, one respondent suggested that EIA pull
information from state databases. Another respondent suggested EIA provide respondents with their
previously submitted survey forms. The other respondent did not have any suggestions, but did say the
county level change was a great start to improving the survey.
DISCUSSION
Interviews with respondents revealed similar feelings regarding the EIA-23L survey. The common
sentiment amongst respondents is that the survey is too time consuming. Respondents also mentioned
that the RIGS online tool is difficult to use and a reason why the survey takes a significant amount of
time to complete. When asked how EIA could improve the survey, most of the respondents did not have
any suggestions, but stated that the change to county level data collection was an improvement.

Findings from Response Burden Interviews for the EIA-23L
November 12, 2015
Page 9 of 15

Appendix A
EIA-23L: Annual Survey of Domestic Oil and Gas Reserves (Field Version)
Response Burden Testing Protocol
October 2015
(This protocol is a guide – the questions presented here won’t necessarily be asked exactly as worded in
the protocol or in this order. It is important to note that not all questions will be asked in every interview.)

Research Goals:


To understand the amount of time is takes a respondent to fill out the EIA-23L survey;



To understand how respondents are calculating their response burden;



And, to understand what factors contribute to a respondent’s response burden.

Part A – Introduction
 Introduce observers and their background
Purpose of visit:
Let me start by telling you a little about what we will be doing today:


We are asking you some questions about your reporting habits for EIA-23L survey.



We are attempting to clear up some response burden issues we have with the EIA-23L form,
specifically how long it takes for a respondent to fill out the survey form.



Ultimately, we are interested in how long it takes you to fill out the survey form and what
elements of the survey take longer to fill out than others.



There are no right or wrong answers, and if something doesn’t make sense to you or you have
any questions, please ask.



Before we get started, I'd like to audio record this interview so I don't have to later rely on my
memory. This session is confidential, which means only persons directly related to this project
can listen to your tapes, and the recordings are erased once our report is written. The report
combines information from all of our interviews and contains no information that personally
identifies you, the mine, or the company at which you work. Is it okay if I record this interview?



Do you have any questions?

Part B – Background Information
Let’s begin by talking about your establishment(s):


How long has your company been operating oil and gas wells?

What is your role at this company?

Findings from Response Burden Interviews for the EIA-23L
November 12, 2015
Page 10 of 15



What is your title?



What are your responsibilities?



Did you complete the 23L survey form last year?



How many years have you been filling out the 23L?

Part B – EIA-23L Response Burden
As you may or may not have heard, EIA is going to make some changes to the 23L. One of the biggest
changes we are looking to make is to no longer collect data at the field level, and instead collect data at
the county level.


Based on the current version of the form, how much time does it take for you to complete the EIA23L?



How much time do you spend collecting and reporting the 2014 information for the most recent
EIA-23L survey?



Do you access the data needed for the survey by yourself or do you have to work with others to
gather data? If you received the data from others, how many other people do you need to
coordinate with to compile the information to report on the form?



Do you fill out the survey form manually or automatically? (Using a program)



Now, if your company reported reserves at the county level instead of by field, how much time do
you estimate it will take to gather and complete the EIA-23L reporting at a county level?
o

If needed, would you say that it will increase or decrease the amount of time it takes to fill
the form out, or would you say it will take the same amount of time?



Do you consider the change in reporting requirements easier or more difficult?



Do you have any start-up costs, such purchasing new software or equipment, to report your
company’s information at the county level instead of the field level?

Part C- Wrap up


Do you complete any other survey reports for EIA?



If so, how does completing the EIA-23L compare with any of the other EIA oil and gas surveys
that you complete?
Findings from Response Burden Interviews for the EIA-23L
November 12, 2015
Page 11 of 15



Do you have any suggestions on how we can make it easier for your company to report on this
survey form?

Findings from Response Burden Interviews for the EIA-23L
November 12, 2015
Page 12 of 15

Appendix B
PART B RESPONSE BURDEN

CURRENT SURVEY
LARGE COMPANIES
GATHER

COMPILE TOTAL
10
2
12
48
112
160
60
20
80

PROPOSED SURVEY
LARGE COMPANIES
GATHER

MEDIUM COMPANIES
MEDIUM COMPANIES
GATHER
COMPILE TOTAL
GATHER
19
1
20
80
80
160
3
117
120
SMALL COMPANIES
GATHER

COMPILE TOTAL
7
1
8
40
120
160
3
1
4

COMPILE TOTAL
10
1
11
24
96
120
24
16
40

COMPILE TOTAL
9
1
10
80
40
120
3
117
120

SMALL COMPANIES
GATHER

COMPILE TOTAL
3
1
4
"GREATLY REDUCED"
<120
<160
“MORE DIFFICULT INITIALLY”
>1
>4

*FIGURES ARE IN
HOURS

RESPONSE BURDEN FIELD LEVEL SURVEY






LARGE COMPANIES
All of the respondents for large companies reported they completed the form without the
assistance of others
1 of the 3 respondents for large companies reported filling out the survey manually
2 of the 3 respondents for large companies reported they used a program to fill out the survey
MEDIUM COMPANIES
2 of the 3 respondents for medium companies stated they completed the survey form with the
assistance of others
Findings from Response Burden Interviews for the EIA-23L
November 12, 2015
Page 13 of 15






2 of the 3 respondents for medium companies stated that they fill out the survey form manually
SMALL COMPANIES
2 of the 3 respondents for small companies stated they completed the survey form without the
assistance of others
2 of the 3 respondents for small companies stated they fill out the survey form manually

ESTIMATED RESPONSE BURDEN COUNTY LEVEL SURVEY













LARGE COMPANIES
None of the respondents for large companies stated they would need to purchase new software
or equipment to fill out the new EIA-23L survey. However, one respondent noted having to
modify the program used to fill out the survey automatically
All of the respondents for large companies stated that the change to county level collection
would make the survey easier to fill out

MEDIUM COMPANIES
2 of the 3 respondents for medium companies stated that the change to county level collection
would make the survey easier to fill out
1 of the 3 respondents for medium companies stated that the change to county level collection
would make the survey more difficult to fill out
o This respondent stated their current data program is field level based and this is would
cause issues. This respondent could not quantify the increase in time associate with the
change.
None of the respondents for medium companies reported that they would need to purchase
new software or equipment to fill out the new EIA-23L survey
SMALL COMPANIES
2 of the 3 respondents for small companies stated that the change to county level collection
would make the survey easier to fill out
1 of the 3 respondents for small companies stated that the change to county level would make
the survey more difficult to fill out, but “eventually it will be ok”
None of the respondents for small companies reported that they will need to purchase new
software or equipment to complete the new survey

Part C WRAP UP


LARGE COMPANIES
2 of the 3 respondents for large companies commented on the RIGS online tool for the EIA-23L
survey
o 1 of the 2 respondents described the online tool as “archaic”
Findings from Response Burden Interviews for the EIA-23L
November 12, 2015
Page 14 of 15


















o 1 of the 2 respondents described the online tool as “tedious to use”
1 of the 3 respondents for large companies described the EIA-23L as “straight forward”
1 of the 3 respondents for large companies suggested adding last year’s company data to the
EIA-23L survey tool
MEDIUM COMPANIES
2 of the 3 respondents for medium companies commented on the RIGS online tool
o 1 of the 2 respondents stated the online tool took a lot of time to use and changing data
in the survey was difficult
o 1 of the 2 respondents stated the online tool has issues that need to be addressed
1 of the 3 respondents for medium companies commented that the EIA-23 survey was
cumbersome
1 of the 3 respondents for medium companies commented that the EIA-23 survey was “not
terrible” and compared it to filing taxes
1 of the 3 respondents for medium companies had suggestions for improving the EIA-23L
survey. This respondent stated the RIGS online tool has programming issues that need to be
addressed
2 of the 3 respondents for medium companies did not have suggestion for improving the EIA23L survey, but 1 of these 2 respondents stated the move was “a step in the right direction”
SMALL COMPANIES
1 of the 3 respondents for small companies described the survey as “time consuming”
1 of the 3 respondents for small companies describe it as “Not the top priority, but not bad”
2 of the 3 respondents for small sized companies had suggestions for how to improve the EIA23L survey
o 2 of the 2 respondents suggested making the previously submitted survey available to
the companies that submitted them
o 1 of the 2 respondents suggested providing clearer definitions on the survey form
o 1 of the 2 respondents suggested populating the RIGS online tool with last year’s data
o 1 of the 2 respondents suggested EIA pull information form state databases
1 of the 3 respondents for small companies did not have suggestions for how to improve the
EIA-23L survey, but did state the move to county level data collection was a great start

Findings from Response Burden Interviews for the EIA-23L
November 12, 2015
Page 15 of 15