Pretest Report
Pre-tests were conducted with 9 individuals over the period 12-15 April 2016. The goals of conducting these pre-tests were to test the experimental procedures, instructions, and interface; and to observe a distribution of payments and completion times. Specifically, the pre-tests were intended to:
Discover any bugs that might occur in the experimental software. Result: Minor software bugs identified and corrected with layout of menus in the Cumulative Prospect Theory (CPT) elicitation task.
Collect subject feedback on aspects of instructions that were unclear during the pretest. Result: Feedback received and instructions updated accordingly.
Test the total running time of the experiment, including completing informed consent, reading instructions, completing all experimental rounds, completing demographic questionnaire, and paying subjects. Result: Session lengths ranged from 47-74 minutes, with median run time of 62 minutes.
Test the distribution of subject payments. Result: Payments ranged from $10 - $29.80, with median payment $23.33.
Three pre-tests were conducted at the Department of Environmental and Natural Resource Economics’ Policy Simulation Laboratory (SimLab) at the University of Rhode Island. Pre-tests were conducted over the period 12-15 April 2016 and involved 2-5 individuals per session. Participants were undergraduate and graduate students from the University of Rhode Island. Over all three sessions, 9 individuals participated in the pre-tests.
Pre-test procedures included completing informed consent, reading instructions, completing all experimental rounds, completing demographic questionnaire, and paying subjects. All pre-test subjects completed the full set of pre-test procedures. Additionally, subjects were invited to provide feedback on the experimental procedures and instructions after the pre-test was completed.
The table below displays the session length and payment received for all 9 pre-test participants:
Subject ID |
Session Length (minutes) |
Total Payout Earned |
Equivalent Hourly Payment Rate |
1 |
62 |
$18.00 |
$17.42 |
2 |
73 |
$29.80 |
$24.49 |
3 |
55 |
$10.00 |
$10.91 |
4 |
74 |
$26.23 |
$21.27 |
5 |
63 |
$13.00 |
$12.38 |
6 |
58 |
$24.28 |
$25.12 |
7 |
47 |
$14.00 |
$17.87 |
8 |
59 |
$23.33 |
$23.72 |
9 |
65 |
$27.11 |
$25.02 |
Mean |
61.78 |
$20.64 |
$19.80 |
Median |
62 |
$23.33 |
$21.27 |
All subjects were requested to provide feedback about the experimental procedures and instructions once the pre-test was completed.
Multiple subjects reported that menus were not consistently aligned across rounds in the CPT elicitation task. In a few rows in some of the menus, the boxes were split over two rows instead of being aligned in a single row. This bug has since been identified and corrected.
Three subjects requested opportunities to practice using the interface before starting each new task. We have since added a place to do this on the instruction page for each task.
Two subjects requested additional description in the instructions about how to interpret Option A and Option B in the CPT elicitation task. We have since provided additional text in the instructions for this task to illustrate how payments and probabilities are linked in the visual display.
Supporting Statement: Attachment H Page
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Rosch, Stephanie - ERS |
| File Modified | 0000-00-00 |
| File Created | 2021-01-23 |