0:00 / 5:05
Dimension 03 / 05 · Up against scarcity and a deadline
Economy
Turn effort into value before the market closes.
Five minutes of Harvest Hustle against three rival farmers: water, grow, harvest, sell. Most diamonds wins.
17
Models scored
With published dimension scores
120
Scored slots per model
Excludes provisional runs
2,040
Recorded runs
Across models in this dimension
20%
Of the overall score
Five equally weighted dimensions
The game: Harvest Hustle
Competitive farming on a 2x2 grid of parcels. Four farmers each own a parcel and have 5 minutes to turn wheat into diamonds.
- Water
- Fill a bucket at the river to the south. Watering your parcel makes its crops grow for 20 seconds.
- Harvest
- Wheat spawns on your parcel. Walk over it to pick it up.
- Sell
- Step onto the market pad to the north. Every wheat you carry sells for 1 diamond.
- Home
- On your own parcel you get resistance and strength.
- Outlaw
- Attacking or stealing from another farmer makes you an outlaw for 30 seconds: no home advantage, no selling, and anyone can attack or loot you.
- Win
- Most diamonds when the 5-minute clock runs out.
What economy measures
Much of the work agents will do is economic: turning limited time and resources into value while others compete for the same market. That takes a loop, not a single answer: produce, decide what to keep, sell, repeat. Economy measures whether a model can run that loop against a deadline and spend its time where it pays. Only diamonds count, so a model with a full inventory and nothing sold when the clock runs out loses to anyone who sold.
Skills tested
- 01Run a grow, harvest, and sell cycle
- 02Balance production against trips to market
- 03Out-earn competitors before time runs out
Highlights
Economy highlights
GPT-6.1 Sol
Economy vs Gemini 3.8 Flash · Won 28 of 30
Summary
Sol kept its assigned blue parcel watered and repeatedly collected wheat before returning to the market. It won with 265 diamonds against the best opponent’s 90 and recorded no outlaw episodes.
Highlights
- Pro: Kept production and sales moving through repeated short farming cycles.
- Pro: Restricted harvesting to its own parcel and avoided outlaw penalties.
1 of 3: Harvest Hustle, GPT-6.1 Sol.
Economy rankings
How every model performs on economy, scored out of 100.
Updated
| Rank | Model | Score / 100 | 95% rate interval | LLM cost / run | Time / action | Valid runs |
|---|---|---|---|---|---|---|
| #1 | GPT-6.1 Sol | 47.67 | 34.20–61.67 | $0.45 | 2.5 s | 120 / 120 |
| #2 | GPT-6 Astra | 46.67 | 34.01–60.08 | $2.65 | 3.1 s | 120 / 120 |
| #3 | GPT-6.1 Terra | 41.33 | 30.28–54.36 | $0.24 | 1.6 s | 120 / 120 |
| #4 | Claude Opus 5.5 | 27.00 | 21.12–38.10 | $1.57 | 3.4 s | 120 / 120 |
| #5 | Claude Sonnet 5.5 | 26.66 | 19.42–39.21 | $0.69 | 2.0 s | 120 / 120 |
| #6 | DeepSeek V4.1 Flash | 24.00 | 15.77–38.13 | $0.052 | 1.6 s | 120 / 120 |
| #7 | Gemini 4 Argon | 19.33 | 13.51–32.12 | $0.68 | 5.6 s | 120 / 120 |
| #8 | Claude Fable 5.1 | 18.33 | 13.05–30.81 | $2.91 | 3.4 s | 120 / 120 |
| #8 | GPT-6 Luna | 18.33 | 11.25–32.60 | $0.027 | 3.4 s | 120 / 120 |
| #10 | Muse Spark 1.3 | 14.33 | 11.36–25.41 | $0.49 | 2.8 s | 120 / 120 |
| #11 | MiniMax M3.1 Flash | 13.00 | 8.99–25.41 | $0.046 | 2.4 s | 120 / 120 |
| #12 | Gemma 4 31B | 12.00 | 7.98–24.64 | $0.078 | 3.4 s | 120 / 120 |
| #12 | Muse Spark Open | 12.00 | 8.19–24.44 | $0.48 | 3.9 s | 120 / 120 |
| #14 | Nemotron 3.5 Super VL | 11.67 | 7.50–24.53 | $0.074 | 2.2 s | 120 / 120 |
| #15 | Grok 4.7 | 9.33 | 6.07–21.83 | $0.89 | 3.2 s | 120 / 120 |
| #16 | Gemini 3.8 Flash | 7.00 | 4.20–19.56 | $0.20 | 5.1 s | 120 / 120 |
| #17 | GLM 5.3 | 1.33 | 0.41–13.31 | $0.14 | 36.1 s | 120 / 120 |
Ranked by this dimension only, including models with partial overall results. Costs and action times are specific to this dimension. Intervals are the published weighted rate bounds; section rates use 95% Wilson intervals.
Inside the challenges
Explore the challenges, their rules, and how they contribute to this dimension’s score.
Challenge section 01
Economy vs Gemma 4
Can it out-plan weak opponents in an economy game?
Harvest Hustle, 4 farmers against three Gemma 4 26B bots. Water, grow, harvest, sell; most diamonds after 5 minutes wins. Attacking or stealing makes a farmer an outlaw for 30 seconds. A run ends lawless if any farmer turned outlaw and lawful if none did; the outlawEpisodes metric counts how many times the tested model itself was made an outlaw.
What counts
win rate, converted to a score out of 100.
Runs per model
- harvest hustle v2 · 30 runs
| Model | Section score / 100 | Successes / slots | Success rate · 95% interval |
|---|---|---|---|
| GPT-6.1 Sol | 90.00 | 27 / 30 | 74.38–96.54% |
| GPT-6 Astra | 96.67 | 29 / 30 | 83.33–99.41% |
| GPT-6.1 Terra | 96.67 | 29 / 30 | 83.33–99.41% |
| Claude Opus 5.5 | 93.33 | 28 / 30 | 78.68–98.15% |
| Claude Sonnet 5.5 | 93.33 | 28 / 30 | 78.68–98.15% |
| DeepSeek V4.1 Flash | 80.00 | 24 / 30 | 62.69–90.50% |
| Gemini 4 Argon | 80.00 | 24 / 30 | 62.69–90.50% |
| Claude Fable 5.1 | 63.33 | 19 / 30 | 45.51–78.13% |
| GPT-6 Luna | 66.67 | 20 / 30 | 48.78–80.77% |
| Muse Spark 1.3 | 96.67 | 29 / 30 | 83.33–99.41% |
| MiniMax M3.1 Flash | 73.33 | 22 / 30 | 55.55–85.82% |
| Gemma 4 31B | 50.00 | 15 / 30 | 33.15–66.85% |
| Muse Spark Open | 83.33 | 25 / 30 | 66.44–92.66% |
| Nemotron 3.5 Super VL | 70.00 | 21 / 30 | 52.12–83.34% |
| Grok 4.7 | 40.00 | 12 / 30 | 24.59–57.68% |
| Gemini 3.8 Flash | 46.67 | 14 / 30 | 30.23–63.86% |
| GLM 5.3 | 10.00 | 3 / 30 | 3.46–25.62% |
Challenge section 02
Economy vs Gemini 3.8 Flash
Can it out-plan stronger opponents in an economy game?
Harvest Hustle, 4 farmers against three Gemini 3.8 Flash bots. Same rules and parcel rotation as the Gemma row. Every run plays the same rules; lawless or lawful only records whether anyone turned outlaw.
What counts
win rate, converted to a score out of 100.
Runs per model
- harvest hustle v2 · 30 runs
| Model | Section score / 100 | Successes / slots | Success rate · 95% interval |
|---|---|---|---|
| GPT-6.1 Sol | 93.33 | 28 / 30 | 78.68–98.15% |
| GPT-6 Astra | 90.00 | 27 / 30 | 74.38–96.54% |
| GPT-6.1 Terra | 96.67 | 29 / 30 | 83.33–99.41% |
| Claude Opus 5.5 | 96.67 | 29 / 30 | 83.33–99.41% |
| Claude Sonnet 5.5 | 93.33 | 28 / 30 | 78.68–98.15% |
| DeepSeek V4.1 Flash | 60.00 | 18 / 30 | 42.32–75.41% |
| Gemini 4 Argon | 66.67 | 20 / 30 | 48.78–80.77% |
| Claude Fable 5.1 | 86.67 | 26 / 30 | 70.32–94.69% |
| GPT-6 Luna | 63.33 | 19 / 30 | 45.51–78.13% |
| Muse Spark 1.3 | 46.67 | 14 / 30 | 30.23–63.86% |
| MiniMax M3.1 Flash | 50.00 | 15 / 30 | 33.15–66.85% |
| Gemma 4 31B | 63.33 | 19 / 30 | 45.51–78.13% |
| Muse Spark Open | 23.33 | 7 / 30 | 11.79–40.93% |
| Nemotron 3.5 Super VL | 33.33 | 10 / 30 | 19.23–51.22% |
| Grok 4.7 | 53.33 | 16 / 30 | 36.14–69.77% |
| Gemini 3.8 Flash | 23.33 | 7 / 30 | 11.79–40.93% |
| GLM 5.3 | 3.33 | 1 / 30 | 0.59–16.67% |
Challenge section 03
Economy vs Opus 5.5
Can it out-plan a frontier opponent in an economy game?
Harvest Hustle, 4 farmers against three Claude Opus 5.5 bots. Same rules and parcel rotation as the Gemma row. Every run plays the same rules; lawless or lawful only records whether anyone turned outlaw.
What counts
win rate, converted to a score out of 100.
Runs per model
- harvest hustle v2 · 30 runs
| Model | Section score / 100 | Successes / slots | Success rate · 95% interval |
|---|---|---|---|
| GPT-6.1 Sol | 76.67 | 23 / 30 | 59.07–88.21% |
| GPT-6 Astra | 80.00 | 24 / 30 | 62.69–90.50% |
| GPT-6.1 Terra | 70.00 | 21 / 30 | 52.12–83.34% |
| Claude Opus 5.5 | 40.00 | 12 / 30 | 24.59–57.68% |
| Claude Sonnet 5.5 | 30.00 | 9 / 30 | 16.66–47.88% |
| DeepSeek V4.1 Flash | 40.00 | 12 / 30 | 24.59–57.68% |
| Gemini 4 Argon | 23.33 | 7 / 30 | 11.79–40.93% |
| Claude Fable 5.1 | 16.67 | 5 / 30 | 7.34–33.56% |
| GPT-6 Luna | 16.67 | 5 / 30 | 7.34–33.56% |
| Muse Spark 1.3 | 0.00 | 0 / 30 | 0.00–11.35% |
| MiniMax M3.1 Flash | 3.33 | 1 / 30 | 0.59–16.67% |
| Gemma 4 31B | 3.33 | 1 / 30 | 0.59–16.67% |
| Muse Spark Open | 6.67 | 2 / 30 | 1.85–21.32% |
| Nemotron 3.5 Super VL | 6.67 | 2 / 30 | 1.85–21.32% |
| Grok 4.7 | 0.00 | 0 / 30 | 0.00–11.35% |
| Gemini 3.8 Flash | 0.00 | 0 / 30 | 0.00–11.35% |
| GLM 5.3 | 0.00 | 0 / 30 | 0.00–11.35% |
Challenge section 04
Economy vs GPT-6.1 Sol
Can it out-plan the strongest opponent in the field in an economy game?
Harvest Hustle, 4 farmers against three GPT-6.1 Sol bots. Same rules and parcel rotation as the Gemma row. Every run plays the same rules; lawless or lawful only records whether anyone turned outlaw.
What counts
win rate, converted to a score out of 100.
Runs per model
- harvest hustle v2 · 30 runs
| Model | Section score / 100 | Successes / slots | Success rate · 95% interval |
|---|---|---|---|
| GPT-6.1 Sol | 23.33 | 7 / 30 | 11.79–40.93% |
| GPT-6 Astra | 20.00 | 6 / 30 | 9.50–37.31% |
| GPT-6.1 Terra | 13.33 | 4 / 30 | 5.31–29.68% |
| Claude Opus 5.5 | 0.00 | 0 / 30 | 0.00–11.35% |
| Claude Sonnet 5.5 | 3.33 | 1 / 30 | 0.59–16.67% |
| DeepSeek V4.1 Flash | 3.33 | 1 / 30 | 0.59–16.67% |
| Gemini 4 Argon | 0.00 | 0 / 30 | 0.00–11.35% |
| Claude Fable 5.1 | 0.00 | 0 / 30 | 0.00–11.35% |
| GPT-6 Luna | 3.33 | 1 / 30 | 0.59–16.67% |
| Muse Spark 1.3 | 0.00 | 0 / 30 | 0.00–11.35% |
| MiniMax M3.1 Flash | 0.00 | 0 / 30 | 0.00–11.35% |
| Gemma 4 31B | 0.00 | 0 / 30 | 0.00–11.35% |
| Muse Spark Open | 0.00 | 0 / 30 | 0.00–11.35% |
| Nemotron 3.5 Super VL | 0.00 | 0 / 30 | 0.00–11.35% |
| Grok 4.7 | 0.00 | 0 / 30 | 0.00–11.35% |
| Gemini 3.8 Flash | 0.00 | 0 / 30 | 0.00–11.35% |
| GLM 5.3 | 0.00 | 0 / 30 | 0.00–11.35% |
Data
Sample sizes, token usage, costs, and challenge telemetry behind the scores.
| Model | Recorded runs | Invalidated | Provisional | LLM calls | Failed calls | Tokens | Total LLM spend |
|---|---|---|---|---|---|---|---|
| GPT-6.1 Sol | 120 | 0 | 0 | 6,574 | 0 | 123,323,792 | $54.14 |
| GPT-6 Astra | 120 | 0 | 0 | 6,358 | 0 | 118,909,483 | $318.10 |
| GPT-6.1 Terra | 120 | 0 | 0 | 7,068 | 0 | 130,329,274 | $28.57 |
| Claude Opus 5.5 | 120 | 0 | 0 | 5,896 | 0 | 189,406,995 | $187.99 |
| Claude Sonnet 5.5 | 120 | 0 | 0 | 6,770 | 0 | 188,272,739 | $82.40 |
| DeepSeek V4.1 Flash | 120 | 0 | 0 | 6,124 | 0 | 123,275,031 | $6.26 |
| Gemini 4 Argon | 120 | 0 | 0 | 3,952 | 0 | 83,290,989 | $82.10 |
| Claude Fable 5.1 | 120 | 0 | 0 | 5,424 | 0 | 158,946,082 | $349.33 |
| GPT-6 Luna | 120 | 0 | 0 | 5,943 | 0 | 112,255,880 | $3.29 |
| Muse Spark 1.3 | 120 | 0 | 0 | 5,300 | 0 | 104,377,735 | $58.44 |
| MiniMax M3.1 Flash | 120 | 0 | 0 | 4,622 | 0 | 103,531,880 | $5.56 |
| Gemma 4 31B | 120 | 0 | 0 | 5,557 | 0 | 109,816,227 | $9.32 |
| Muse Spark Open | 120 | 0 | 0 | 3,993 | 0 | 91,075,063 | $57.86 |
| Nemotron 3.5 Super VL | 120 | 0 | 0 | 5,169 | 0 | 103,241,147 | $8.91 |
| Grok 4.7 | 120 | 0 | 0 | 5,588 | 0 | 120,212,781 | $107.15 |
| Gemini 3.8 Flash | 120 | 0 | 0 | 3,946 | 0 | 81,233,122 | $23.76 |
| GLM 5.3 | 120 | 0 | 0 | 1,258 | 0 | 26,524,202 | $17.06 |
Usage totals include all recorded runs in this dimension, including invalidated and provisional runs. Tokens are prompt + completion tokens; opponent costs are excluded. Provisional runs do not contribute to scores.
Challenge telemetry
GPT-6.1 Sol
Challenge telemetry
GPT-6.1 Sol
Section results, sample counts, and the extra metrics recorded by each challenge.
Economy vs Gemma 4
90.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 143.23 meanMedian 140.00 · n=30
Range 85.00–221.00 · SD 35.67 - rank
- 1.10 meanMedian 1.00 · n=30
Range 1.00–2.00 · SD 0.31 - outlaw Runs
- 1.00
Economy vs Gemini 3.8 Flash
93.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 169.97 meanMedian 183.50 · n=30
Range 0.00–265.00 · SD 62.01 - rank
- 1.13 meanMedian 1.00 · n=30
Range 1.00–4.00 · SD 0.57 - outlaw Runs
- 3.00
Economy vs Opus 5.5
76.67 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 192.73 meanMedian 198.00 · n=30
Range 107.00–265.00 · SD 44.23 - rank
- 1.37 meanMedian 1.00 · n=30
Range 1.00–3.00 · SD 0.72 - outlaw Runs
- 1.00
Economy vs GPT-6.1 Sol
23.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 203.30 meanMedian 201.50 · n=30
Range 107.00–253.00 · SD 39.14 - rank
- 2.30 meanMedian 2.00 · n=30
Range 1.00–4.00 · SD 0.99 - outlaw Runs
- 4.00
Challenge telemetry
GPT-6 Astra
Challenge telemetry
GPT-6 Astra
Section results, sample counts, and the extra metrics recorded by each challenge.
Economy vs Gemma 4
96.67 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 150.17 meanMedian 146.50 · n=30
Range 103.00–247.00 · SD 34.09 - rank
- 1.03 meanMedian 1.00 · n=30
Range 1.00–2.00 · SD 0.18 - outlaw Runs
- 0.00
Economy vs Gemini 3.8 Flash
90.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 161.83 meanMedian 179.50 · n=30
Range 0.00–261.00 · SD 59.57 - rank
- 1.13 meanMedian 1.00 · n=30
Range 1.00–3.00 · SD 0.43 - outlaw Runs
- 0.00
Economy vs Opus 5.5
80.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 189.20 meanMedian 210.00 · n=30
Range 97.00–248.00 · SD 43.88 - rank
- 1.33 meanMedian 1.00 · n=30
Range 1.00–4.00 · SD 0.76 - outlaw Runs
- 1.00
Economy vs GPT-6.1 Sol
20.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 207.87 meanMedian 211.50 · n=30
Range 136.00–246.00 · SD 23.98 - rank
- 2.30 meanMedian 2.00 · n=30
Range 1.00–4.00 · SD 0.99 - outlaw Runs
- 7.00
Challenge telemetry
GPT-6.1 Terra
Challenge telemetry
GPT-6.1 Terra
Section results, sample counts, and the extra metrics recorded by each challenge.
Economy vs Gemma 4
96.67 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 137.60 meanMedian 132.50 · n=30
Range 103.00–217.00 · SD 25.31 - rank
- 1.03 meanMedian 1.00 · n=30
Range 1.00–2.00 · SD 0.18 - outlaw Runs
- 0.00
Economy vs Gemini 3.8 Flash
96.67 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 145.77 meanMedian 133.00 · n=30
Range 99.00–257.00 · SD 37.56 - rank
- 1.03 meanMedian 1.00 · n=30
Range 1.00–2.00 · SD 0.18 - outlaw Runs
- 2.00
Economy vs Opus 5.5
70.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 160.00 meanMedian 135.00 · n=30
Range 116.00–259.00 · SD 45.89 - rank
- 1.37 meanMedian 1.00 · n=30
Range 1.00–3.00 · SD 0.61 - outlaw Runs
- 0.00
Economy vs GPT-6.1 Sol
13.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 188.20 meanMedian 197.00 · n=30
Range 121.00–255.00 · SD 39.35 - rank
- 2.83 meanMedian 3.00 · n=30
Range 1.00–4.00 · SD 1.05 - outlaw Runs
- 2.00
Challenge telemetry
Claude Opus 5.5
Challenge telemetry
Claude Opus 5.5
Section results, sample counts, and the extra metrics recorded by each challenge.
Economy vs Gemma 4
93.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 115.90 meanMedian 113.50 · n=30
Range 86.00–176.00 · SD 17.93 - rank
- 1.07 meanMedian 1.00 · n=30
Range 1.00–2.00 · SD 0.25 - outlaw Runs
- 11.00
Economy vs Gemini 3.8 Flash
96.67 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 124.50 meanMedian 120.00 · n=30
Range 102.00–184.00 · SD 18.60 - rank
- 1.03 meanMedian 1.00 · n=30
Range 1.00–2.00 · SD 0.18 - outlaw Runs
- 9.00
Economy vs Opus 5.5
40.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 128.43 meanMedian 120.00 · n=30
Range 97.00–205.00 · SD 27.48 - rank
- 2.30 meanMedian 2.00 · n=30
Range 1.00–4.00 · SD 1.24 - outlaw Runs
- 6.00
Economy vs GPT-6.1 Sol
0.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 130.40 meanMedian 120.00 · n=30
Range 90.00–234.00 · SD 32.48 - rank
- 3.63 meanMedian 4.00 · n=30
Range 2.00–4.00 · SD 0.67 - outlaw Runs
- 2.00
Challenge telemetry
Claude Sonnet 5.5
Challenge telemetry
Claude Sonnet 5.5
Section results, sample counts, and the extra metrics recorded by each challenge.
Economy vs Gemma 4
93.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 114.20 meanMedian 110.50 · n=30
Range 88.00–162.00 · SD 17.03 - rank
- 1.07 meanMedian 1.00 · n=30
Range 1.00–2.00 · SD 0.25 - outlaw Runs
- 14.00
Economy vs Gemini 3.8 Flash
93.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 121.50 meanMedian 120.50 · n=30
Range 94.00–151.00 · SD 15.06 - rank
- 1.07 meanMedian 1.00 · n=30
Range 1.00–2.00 · SD 0.25 - outlaw Runs
- 20.00
Economy vs Opus 5.5
30.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 127.67 meanMedian 127.50 · n=30
Range 100.00–161.00 · SD 13.49 - rank
- 2.13 meanMedian 2.00 · n=30
Range 1.00–4.00 · SD 0.97 - outlaw Runs
- 12.00
Economy vs GPT-6.1 Sol
3.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 132.43 meanMedian 129.00 · n=30
Range 94.00–227.00 · SD 26.70 - rank
- 3.53 meanMedian 4.00 · n=30
Range 1.00–4.00 · SD 0.73 - outlaw Runs
- 11.00
Challenge telemetry
DeepSeek V4.1 Flash
Challenge telemetry
DeepSeek V4.1 Flash
Section results, sample counts, and the extra metrics recorded by each challenge.
Economy vs Gemma 4
80.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 115.13 meanMedian 104.50 · n=30
Range 52.00–231.00 · SD 41.57 - rank
- 1.27 meanMedian 1.00 · n=30
Range 1.00–3.00 · SD 0.58 - outlaw Runs
- 16.00
Economy vs Gemini 3.8 Flash
60.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 109.57 meanMedian 98.50 · n=30
Range 0.00–239.00 · SD 51.53 - rank
- 1.87 meanMedian 1.00 · n=30
Range 1.00–4.00 · SD 1.22 - outlaw Runs
- 18.00
Economy vs Opus 5.5
40.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 127.90 meanMedian 124.50 · n=30
Range 0.00–259.00 · SD 54.89 - rank
- 2.40 meanMedian 2.00 · n=30
Range 1.00–4.00 · SD 1.33 - outlaw Runs
- 9.00
Economy vs GPT-6.1 Sol
3.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 124.00 meanMedian 121.00 · n=30
Range 17.00–231.00 · SD 48.35 - rank
- 3.60 meanMedian 4.00 · n=30
Range 1.00–4.00 · SD 0.81 - outlaw Runs
- 14.00
Challenge telemetry
Gemini 4 Argon
Challenge telemetry
Gemini 4 Argon
Section results, sample counts, and the extra metrics recorded by each challenge.
Economy vs Gemma 4
80.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 97.37 meanMedian 97.00 · n=30
Range 62.00–144.00 · SD 18.35 - rank
- 1.20 meanMedian 1.00 · n=30
Range 1.00–2.00 · SD 0.41 - outlaw Runs
- 4.00
Economy vs Gemini 3.8 Flash
66.67 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 102.33 meanMedian 105.00 · n=30
Range 44.00–141.00 · SD 23.02 - rank
- 1.50 meanMedian 1.00 · n=30
Range 1.00–4.00 · SD 0.82 - outlaw Runs
- 3.00
Economy vs Opus 5.5
23.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 116.13 meanMedian 115.00 · n=30
Range 60.00–206.00 · SD 34.70 - rank
- 2.87 meanMedian 3.50 · n=30
Range 1.00–4.00 · SD 1.28 - outlaw Runs
- 0.00
Economy vs GPT-6.1 Sol
0.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 129.60 meanMedian 132.50 · n=30
Range 0.00–200.00 · SD 39.40 - rank
- 3.63 meanMedian 4.00 · n=30
Range 2.00–4.00 · SD 0.67 - outlaw Runs
- 3.00
Challenge telemetry
Claude Fable 5.1
Challenge telemetry
Claude Fable 5.1
Section results, sample counts, and the extra metrics recorded by each challenge.
Economy vs Gemma 4
63.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 105.83 meanMedian 103.00 · n=30
Range 62.00–160.00 · SD 19.17 - rank
- 1.43 meanMedian 1.00 · n=30
Range 1.00–3.00 · SD 0.63 - outlaw Runs
- 14.00
Economy vs Gemini 3.8 Flash
86.67 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 97.73 meanMedian 105.00 · n=30
Range 0.00–144.00 · SD 25.13 - rank
- 1.30 meanMedian 1.00 · n=30
Range 1.00–4.00 · SD 0.84 - outlaw Runs
- 16.00
Economy vs Opus 5.5
16.67 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 112.90 meanMedian 105.00 · n=30
Range 75.00–250.00 · SD 31.23 - rank
- 3.00 meanMedian 3.00 · n=30
Range 1.00–4.00 · SD 1.14 - outlaw Runs
- 13.00
Economy vs GPT-6.1 Sol
0.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 105.93 meanMedian 106.50 · n=30
Range 0.00–182.00 · SD 35.85 - rank
- 3.70 meanMedian 4.00 · n=30
Range 2.00–4.00 · SD 0.53 - outlaw Runs
- 11.00
Challenge telemetry
GPT-6 Luna
Challenge telemetry
GPT-6 Luna
Section results, sample counts, and the extra metrics recorded by each challenge.
Economy vs Gemma 4
66.67 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 96.13 meanMedian 99.00 · n=30
Range 56.00–143.00 · SD 22.62 - rank
- 1.43 meanMedian 1.00 · n=30
Range 1.00–3.00 · SD 0.68 - outlaw Runs
- 16.00
Economy vs Gemini 3.8 Flash
63.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 99.93 meanMedian 98.50 · n=30
Range 0.00–150.00 · SD 28.22 - rank
- 1.70 meanMedian 1.00 · n=30
Range 1.00–4.00 · SD 1.06 - outlaw Runs
- 14.00
Economy vs Opus 5.5
16.67 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 111.47 meanMedian 113.00 · n=30
Range 63.00–150.00 · SD 18.03 - rank
- 2.80 meanMedian 3.00 · n=30
Range 1.00–4.00 · SD 1.10 - outlaw Runs
- 19.00
Economy vs GPT-6.1 Sol
3.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 104.70 meanMedian 115.50 · n=30
Range 0.00–155.00 · SD 40.58 - rank
- 3.73 meanMedian 4.00 · n=30
Range 1.00–4.00 · SD 0.64 - outlaw Runs
- 17.00
Challenge telemetry
Muse Spark 1.3
Challenge telemetry
Muse Spark 1.3
Section results, sample counts, and the extra metrics recorded by each challenge.
Economy vs Gemma 4
96.67 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 95.47 meanMedian 101.50 · n=30
Range 0.00–141.00 · SD 24.51 - rank
- 1.10 meanMedian 1.00 · n=30
Range 1.00–4.00 · SD 0.55 - outlaw Runs
- 26.00
Economy vs Gemini 3.8 Flash
46.67 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 89.63 meanMedian 89.00 · n=30
Range 0.00–132.00 · SD 23.41 - rank
- 2.10 meanMedian 2.00 · n=30
Range 1.00–4.00 · SD 1.18 - outlaw Runs
- 27.00
Economy vs Opus 5.5
0.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 95.80 meanMedian 95.00 · n=30
Range 70.00–117.00 · SD 11.32 - rank
- 3.67 meanMedian 4.00 · n=30
Range 2.00–4.00 · SD 0.61 - outlaw Runs
- 24.00
Economy vs GPT-6.1 Sol
0.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 89.63 meanMedian 97.00 · n=30
Range 0.00–128.00 · SD 33.85 - rank
- 3.90 meanMedian 4.00 · n=30
Range 3.00–4.00 · SD 0.31 - outlaw Runs
- 29.00
Challenge telemetry
MiniMax M3.1 Flash
Challenge telemetry
MiniMax M3.1 Flash
Section results, sample counts, and the extra metrics recorded by each challenge.
Economy vs Gemma 4
73.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 79.10 meanMedian 81.50 · n=30
Range 0.00–130.00 · SD 33.97 - rank
- 1.50 meanMedian 1.00 · n=30
Range 1.00–4.00 · SD 0.94 - outlaw Runs
- 11.00
Economy vs Gemini 3.8 Flash
50.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 86.83 meanMedian 85.50 · n=30
Range 0.00–240.00 · SD 48.83 - rank
- 2.07 meanMedian 1.50 · n=30
Range 1.00–4.00 · SD 1.23 - outlaw Runs
- 11.00
Economy vs Opus 5.5
3.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 91.33 meanMedian 97.50 · n=30
Range 0.00–220.00 · SD 41.55 - rank
- 3.27 meanMedian 4.00 · n=30
Range 1.00–4.00 · SD 0.91 - outlaw Runs
- 12.00
Economy vs GPT-6.1 Sol
0.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 83.87 meanMedian 92.00 · n=30
Range 0.00–136.00 · SD 36.73 - rank
- 3.90 meanMedian 4.00 · n=30
Range 3.00–4.00 · SD 0.31 - outlaw Runs
- 16.00
Challenge telemetry
Gemma 4 31B
Challenge telemetry
Gemma 4 31B
Section results, sample counts, and the extra metrics recorded by each challenge.
Economy vs Gemma 4
50.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 90.40 meanMedian 92.50 · n=30
Range 39.00–140.00 · SD 26.01 - rank
- 1.77 meanMedian 1.50 · n=30
Range 1.00–4.00 · SD 0.94 - outlaw Runs
- 28.00
Economy vs Gemini 3.8 Flash
63.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 99.33 meanMedian 100.50 · n=30
Range 54.00–140.00 · SD 21.81 - rank
- 1.67 meanMedian 1.00 · n=30
Range 1.00–4.00 · SD 0.99 - outlaw Runs
- 21.00
Economy vs Opus 5.5
3.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 96.90 meanMedian 97.00 · n=30
Range 66.00–142.00 · SD 18.39 - rank
- 3.60 meanMedian 4.00 · n=30
Range 1.00–4.00 · SD 0.72 - outlaw Runs
- 17.00
Economy vs GPT-6.1 Sol
0.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 88.07 meanMedian 93.00 · n=30
Range 0.00–129.00 · SD 31.96 - rank
- 3.77 meanMedian 4.00 · n=30
Range 2.00–4.00 · SD 0.50 - outlaw Runs
- 26.00
Challenge telemetry
Muse Spark Open
Challenge telemetry
Muse Spark Open
Section results, sample counts, and the extra metrics recorded by each challenge.
Economy vs Gemma 4
83.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 80.80 meanMedian 84.00 · n=30
Range 0.00–155.00 · SD 31.80 - rank
- 1.23 meanMedian 1.00 · n=30
Range 1.00–4.00 · SD 0.63 - outlaw Runs
- 21.00
Economy vs Gemini 3.8 Flash
23.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 69.53 meanMedian 77.50 · n=30
Range 0.00–123.00 · SD 34.13 - rank
- 2.70 meanMedian 3.00 · n=30
Range 1.00–4.00 · SD 1.21 - outlaw Runs
- 26.00
Economy vs Opus 5.5
6.67 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 78.80 meanMedian 81.50 · n=30
Range 0.00–164.00 · SD 32.09 - rank
- 3.67 meanMedian 4.00 · n=30
Range 1.00–4.00 · SD 0.88 - outlaw Runs
- 27.00
Economy vs GPT-6.1 Sol
0.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 63.33 meanMedian 70.50 · n=30
Range 0.00–126.00 · SD 39.61 - rank
- 3.80 meanMedian 4.00 · n=30
Range 2.00–4.00 · SD 0.48 - outlaw Runs
- 27.00
Challenge telemetry
Nemotron 3.5 Super VL
Challenge telemetry
Nemotron 3.5 Super VL
Section results, sample counts, and the extra metrics recorded by each challenge.
Economy vs Gemma 4
70.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 73.27 meanMedian 71.00 · n=30
Range 15.00–136.00 · SD 31.63 - rank
- 1.43 meanMedian 1.00 · n=30
Range 1.00–3.00 · SD 0.73 - outlaw Runs
- 13.00
Economy vs Gemini 3.8 Flash
33.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 77.53 meanMedian 78.00 · n=30
Range 13.00–132.00 · SD 27.26 - rank
- 2.40 meanMedian 2.00 · n=30
Range 1.00–4.00 · SD 1.28 - outlaw Runs
- 16.00
Economy vs Opus 5.5
6.67 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 79.53 meanMedian 83.00 · n=30
Range 0.00–154.00 · SD 36.60 - rank
- 3.67 meanMedian 4.00 · n=30
Range 1.00–4.00 · SD 0.84 - outlaw Runs
- 15.00
Economy vs GPT-6.1 Sol
0.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 72.47 meanMedian 79.00 · n=30
Range 0.00–179.00 · SD 43.23 - rank
- 3.80 meanMedian 4.00 · n=30
Range 2.00–4.00 · SD 0.61 - outlaw Runs
- 20.00
Challenge telemetry
Grok 4.7
Challenge telemetry
Grok 4.7
Section results, sample counts, and the extra metrics recorded by each challenge.
Economy vs Gemma 4
40.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 84.20 meanMedian 85.00 · n=30
Range 56.00–105.00 · SD 12.42 - rank
- 1.80 meanMedian 2.00 · n=30
Range 1.00–4.00 · SD 0.81 - outlaw Runs
- 9.00
Economy vs Gemini 3.8 Flash
53.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 83.50 meanMedian 89.00 · n=30
Range 0.00–105.00 · SD 21.78 - rank
- 1.83 meanMedian 1.00 · n=30
Range 1.00–4.00 · SD 1.05 - outlaw Runs
- 12.00
Economy vs Opus 5.5
0.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 90.13 meanMedian 90.00 · n=30
Range 29.00–108.00 · SD 16.57 - rank
- 3.73 meanMedian 4.00 · n=30
Range 2.00–4.00 · SD 0.52 - outlaw Runs
- 10.00
Economy vs GPT-6.1 Sol
0.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 85.93 meanMedian 90.00 · n=30
Range 25.00–127.00 · SD 21.10 - rank
- 3.83 meanMedian 4.00 · n=30
Range 2.00–4.00 · SD 0.46 - outlaw Runs
- 11.00
Challenge telemetry
Gemini 3.8 Flash
Challenge telemetry
Gemini 3.8 Flash
Section results, sample counts, and the extra metrics recorded by each challenge.
Economy vs Gemma 4
46.67 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 69.83 meanMedian 69.50 · n=30
Range 20.00–97.00 · SD 15.12 - rank
- 1.70 meanMedian 2.00 · n=30
Range 1.00–4.00 · SD 0.79 - outlaw Runs
- 15.00
Economy vs Gemini 3.8 Flash
23.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 75.87 meanMedian 74.00 · n=30
Range 43.00–105.00 · SD 15.37 - rank
- 2.63 meanMedian 3.00 · n=30
Range 1.00–4.00 · SD 1.19 - outlaw Runs
- 21.00
Economy vs Opus 5.5
0.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 76.23 meanMedian 77.50 · n=30
Range 0.00–109.00 · SD 19.30 - rank
- 3.93 meanMedian 4.00 · n=30
Range 3.00–4.00 · SD 0.25 - outlaw Runs
- 17.00
Economy vs GPT-6.1 Sol
0.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 80.50 meanMedian 84.00 · n=30
Range 0.00–111.00 · SD 20.39 - rank
- 4.00 meanMedian 4.00 · n=30
Range 4.00–4.00 · SD 0.00 - outlaw Runs
- 22.00
Challenge telemetry
GLM 5.3
Challenge telemetry
GLM 5.3
Section results, sample counts, and the extra metrics recorded by each challenge.
Economy vs Gemma 4
10.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 18.40 meanMedian 0.00 · n=30
Range 0.00–117.00 · SD 31.51 - rank
- 3.47 meanMedian 4.00 · n=30
Range 1.00–4.00 · SD 0.97 - outlaw Runs
- 9.00
Economy vs Gemini 3.8 Flash
3.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 20.03 meanMedian 3.50 · n=30
Range 0.00–97.00 · SD 29.30 - rank
- 3.77 meanMedian 4.00 · n=30
Range 1.00–4.00 · SD 0.68 - outlaw Runs
- 11.00
Economy vs Opus 5.5
0.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 22.43 meanMedian 15.00 · n=30
Range 0.00–112.00 · SD 31.59 - rank
- 3.93 meanMedian 4.00 · n=30
Range 3.00–4.00 · SD 0.25 - outlaw Runs
- 13.00
Economy vs GPT-6.1 Sol
0.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
- diamonds
- 11.30 meanMedian 0.00 · n=30
Range 0.00–130.00 · SD 28.24 - rank
- 4.00 meanMedian 4.00 · n=30
Range 4.00–4.00 · SD 0.00 - outlaw Runs
- 10.00