Mini-supporting Statement Part B

Mini-Supporting Statement for

Alternative tools for improving CRP cost-effectiveness

Section B

OMB Control No. 0536-0070

B. COLLECTION OF INFORMATION USING STATISTICAL METHODS

1. Universe and Sampling

The data collected will be used for research activities which address an important question in conservation program design by comparing alternative mechanisms for competitive purchase (auctions), rather than to produce estimates about the population. The study uses a convenience sample of 192 students recruited from a respondent pool of 4,320 students at UMD, College Park who have expressed interest in participating in economics experiments and who have shared their email addresses with the economics department. Participants will be randomly assigned to 12 experimental sessions to take part in three of five experimental conditions (tight bid cap, loose cap, reference price, endogenous reference price, and grouping). Per the generic clearance, results of the study are not meant to be used to make policy decisions, and the sample is not meant to be nationally representative.

2. Procedures for Collecting Information

Recruitment:

Potential participants will be solicited by an email (Attachment-A: Email for recruitment) from a respondent pool that the UMD, College Park experimental economics lab maintains. The respondent pool is a database of students who have expressed interest in participating in economics experiments at UMD, College Park and who have shared their email addresses with the economics department. Because it is common to get a number of no-shows and last-minute cancellations for a given experiment session, and because it is important to have a particular number of participants in each session, each session will also be “overbooked” by 2 potential participants as backups to ensure there will be the right number of individuals to run the experiment. In cases where more people show up than can be accommodated in a session, any extras who show up before the scheduled start time will receive a $7 payment for time and travel and can reschedule for another session. This is common practice among experimentalists.

Prior to giving their consent, potential participants at the beginning of each experimental session will be provided with: (1) a Consent Form (Attachment F) that contains a description of the study, including details of study purpose, data collection methodology, and burden estimate, as well as the use of the information collected and the voluntary nature of the study; and (2) a copy of the data protection disclaimer (Attachment G).

Experiment:

The goal of this experiment is to investigate the performance of alternative auction mechanisms designed to limit information rents. The performances of four alternative auction designs (aka treatments): loose cap, reference price, endogenous reference price, and grouping will be tested against that of a baseline design (tight bid cap). The experiment is composed of twelve 90-minute sessions (Table 1 – Experimental Design). Participants will be randomly assigned to each experimental session to take part in three of five experimental conditions (reverse auctions) as a bidder. Their earnings will be paid to them in cash at the end of the experiment. The treatment order within each session is random. Each treatment will be used for an average for 12 rounds. But subjects in different sessions often proceed at different speeds, making 15 rounds possible in some circumstances; in all cases each session will maintain a 90 minute maximum experiment time. More details for the experimental design are presented in the Attachment B: Experimental Design.

Once an experimental session is scheduled, it is the responsibility of the respondents to travel to the experiment site. The Experimental Economics Laboratory at UMD is located in Room 4104 Tydings Hall, UMD, College Park.

The auctions are conducted via a Z-tree interface (a specialized computer program) with internet administration. At the beginning of each session and each treatment, participants are given detailed explanations and instructions for both general experimental and specific treatment procedures (Attachment D: Instructions). A short demographic questionnaire (Attachment E: Questionnaire) will also be given to participants during the experiment, for the purposes of collecting control variables useful in basic regression analysis on laboratory experimental data.

Table 1. Experimental Design
Session	Treatment*	Average # of rounds per treatment	Max # of rounds per treatment	Time (in minuties)	# of participants
1	1,2,3	12	15	90	16
2	1,2,4	12	15	90	16
3	1,2,5	12	15	90	16
4	1,3,4	12	15	90	16
5	1,3,5	12	15	90	16
6	1,4,5	12	15	90	16
7	1,2,3	12	15	90	16
8	1,2,4	12	15	90	16
9	1,2,5	12	15	90	16
10	1,3,4	12	15	90	16
11	1,3,5	12	15	90	16
12	1,4,5	12	15	90	16
* Random order of treatment within session.

Debriefing and Payment:

At the end of each session, all respondents will be debriefed. This gives an opportunity to address respondents' questions regarding the study.

Each participant will also receive a cash payment at the end of each 90 minute session. The cash payment will be of uncertain value before the experiments take place, the minimum payment will be $7.00 for “showing up”. Depending on their own strategic behavior, they will be paid based on whether their bid is accepted given the equilibrium in the experimental market. We do not expect any payments in excess of $50 based on the pre-test results; in fact, we believe even 1 payment of $50 is a very low probability. The average payments will be approximately $25. While a maximum cap would be desirable, given that the market equilibriates within the experiment and we are specifically testing a treatment without price caps, we cannot guarantee that someone will not earn more than the $50 if we calibrate the ECUs for a $25 USD average payment.

The payments listed here are for the entire 90 minute session, i.e. all auctions participated in by a given individual. Although individuals participate in many auctions for each auction type, individuals are paid at the end of the 90 minutes based on 2 randomly-drawn rounds for each auction type (for example, in a session with 1 auction treatment that includes 12 rounds, experimentalists will draw two rounds at random. Payment is based on the outcomes within these randomly chosen rounds to prevent any wealth effects from distorting the findings of the experiment¹). This practice is standard in the literature.² Therefore, each participant’s payment will be the sum of 6 randomly drawn rounds: 2 per each auction type. If they earned no money in these 6 rounds, they will receive a 7 USD show up fee.

More detailed explanation for payment method and justifications are given in A.9 of this Mini-supporting Statement.

Power Analysis:

The primary hypotheses to be tested will be univariate comparisons (t-tests and non-parametric tests) of expected cost between each of the four alternative CPR enrollment mechanisms (loose cap, reference price, endogenous reference price, and selection by grouping) and under the baseline auction mechanism (tight bid cap). We seek to detect a minimum detectable effect (MDE) of 40 ECUs (Experimental Currency Units). Our simulations suggest that this MDE is reasonable to expect. Lower effects (effects smaller than 40 ECUs) would certainly be useful to know about and potentially cost-effective to implement in a given practical application. We do not, however, want to pre-commit to running more sessions than are necessary in order to detect a very small effect if, as is the case here, we expect a larger effect.

We used simulations programmed in the computer programming language R to determine the expected cost of procurement for the baseline auction. Simulation results indicate an estimated average of 241.1 ECUs and a standard deviation of 65 for the baseline treatment.

We are interested in the total procurement cost for each auction treatment; we obtain one (non-independent) observation of a given auction outcome each round. Based on our proposed experimental design, each experiment will yield an average of 12 rounds of data per treatment and a total of 36 rounds of data per session (three treatments X 12 rounds = 36). Because the same subjects participate in each of the rounds, the 36 observations generated in each session are not treated as independent. If we have 12 sessions and each session yields 36 observations (rounds), it gives us a total of 12*36 = 432 observations, or an average of 86 (rounded down) observations per auction treatment. Given this plan, the proposed experimental design will support a powerful test (α =.05 and1- β = 0.8) to detect a minimum detectable effect (MDE) of 40.2 for all planned univariate comparisons.

3. Methods to Increase or Maximize Response Rates

Potential participants are sent recruitment emails asking them to take part in an experiment. Many participants will have participated in similar economic experiments conducted on the campus of the University of Maryland. Typically, a single solicitation email is sufficient to recruit enough subjects to complete an experimental session.

Because it is common to get a number of no-shows and last-minute cancellations for a given experiment session, and because it is important to have a particular number of participants in each session, each session will also be “overbooked” by 2 potential participants as backups to ensure there will be the right number of individuals to run the experiment. In cases where more people will show up than can be accommodated for a given session , any extras who show up before the scheduled start time will receive a $7 payment for time and travel and can reschedule for another session. This is common practice among experimentalists.

4. Tests of Procedures or Methods

The methods and procedures in use in this experiment have been used widely in the literature. Procedures for creating a market in a laboratory and carrying out a sealed-bid auction have been known and accepted practice in the experimental economics community for decades (See Davis and Holt 1992, e.g.).

A pre-test was conducted on 9 students in the Experimental Economics Laboratory at UMD, College Park, on 30 June 2014 to test the auction software, its interface, the timing of the experiment and to observe auction ranges and averages. Information on the specific areas of interests being tested, respective testing results, and/or actions taken (if applicable) is summarized below:

1. Discover any bugs that might occur in the experimental software when multiple bidders were interfacing with it simultaneously.

Result: minor software bugs found and fixed.

2. Test how long it took to read experimental instructions and conduct multiple consecutive rounds of each of the five auction treatments. Each of the five auction treatments were run during this pre-test session in order to determine how long it took to conduct a first round of each treatment (the first round of a treatment always takes longer than subsequent rounds, as subjects need time to digest instructions for a new set of auction rules) and to conduct subsequent rounds (the second, third, …, nth round of a treatment take much less time).

Result: The table below summarizes the timing of the pretest and our estimate of total time necessary to read instructions and the number of minutes needed to complete an auction round.

Auction treatment	Time for Instructions (in minutes)	Rounds completed	Time to complete auction rounds (in minutes)	Average time per round (in minutes)
1	8	15	23	1.5
2	0	10	14	1.4
3	8	15	21	1.4
4	6	12	12	1
5	2	12	9	0.75
Average	4.8			1.2

We use these timing results to develop our estimate of how many rounds it is possible to complete during a 90 minute experiment.

3. Test how long it took to complete the experiment – including all rounds – and close the experiment by distributing payment and excusing participants from the lab.

Result: Based on the pretest, we estimate that it takes approximately 10 minutes to seat subjects and prepare them for the experiment and 6 minutes to pay and process subjects at the end of the experiment. We assume that this timing will remain constant throughout the planned experiment, leaving us with 90 – 16 = 74 minutes to read instructions and complete auction rounds.

4. Observe auction ranges and averages.

Result: Average earnings were 45.81 Experimental Currency Units (ECU), with a minimum payment of 9 and a maximum payment of 106. In order to ensure that the average payment is $25, this implies a conversion factor of approximately 0.5. That is, one ECU will be worth $0.50. The high payment in this case would have been $53 and the low payment $4.50. This range might not be representative of the actual experiment when performed with the expected number of participants, however. With more competition (more bidders: 16 instead of 9), the results should become less extreme. It is also possible to change the distribution of opportunity costs or the conversion factor once the experiments begin. That is, after holding one complete session, we could change the parameters of the experiment in order to deal with unexpectedly high (or low) payments.

.

5. Identification of Consultation

The contact individual for this research is Nathaniel Higgins, Ph.D., Economic Research Service, USDA, telephone (202) 694-5602.

The research is being conducted under a cooperative agreement between the Economic Research Service and the University of Maryland. Data will be collected, entered, and provided to ERS by Professor Peter Cramton at the University of Maryland, telephone (240) 479-9345.

REFERENCES

Davis, Douglas D. and Charles A. Holt (1992). Experimental Economics, Princeton University Press, Princeton, NJ.

1 Wealth effects are the theoretical changes in behavior that occur after a given individuals’ wealth increases. Since the CRP is a “one-shot” auction – there is only one CRP auction conducted at a moment in time, not a series of CRP auctions – it is necessary to eliminate wealth effects.

2 See “Incentives in Experiments: A Theoretical Analysis” by Azrieli, Chambers, and Healy. http://www.econ.ucsb.edu/about_us/events/seminar_papers/Healy.pdf.

7