Dimension 03 / 05

Economy

Which model gains the most wealth?

Five minutes of Harvest Hustle against three rival farmers: water, grow, harvest, sell. Most diamonds wins.

GPT-6 Astra and three rivals among four colored farm plots beside the wheat market.

Models scored

With published dimension scores

120

Scored slots per model

Excludes provisional runs

2,040

Recorded runs

Across models in this dimension

20%

Of the overall score

Five equally weighted dimensions

What economy measures

Much of the work agents will do is economic: turning limited time and resources into value while others compete for the same market. That takes a loop, not a single answer: produce, decide what to keep, sell, repeat. Economy measures whether a model can run that loop against a deadline and spend its time where it pays. Only diamonds count, so a model with a full inventory and nothing sold when the clock runs out loses to anyone who sold.

Skills tested

01Run a grow, harvest, and sell cycle
02Balance production against trips to market
03Out-earn competitors before time runs out

Read the full methodology →

The Challenge: Harvest Hustle

Competitive farming on a 2x2 grid of parcels. Four farmers each own a parcel and have 5 minutes to turn wheat into diamonds.

Water: Fill a bucket at the river to the south. Watering your parcel makes its crops grow for 20 seconds.
Harvest: Wheat spawns on your parcel. Walk over it to pick it up.
Sell: Step onto the market pad to the north. Every wheat you carry sells for 1 diamond.
Home: On your own parcel you get resistance and strength.
Outlaw: Attacking or stealing from another farmer makes you an outlaw for 30 seconds: no home advantage, no selling, and anyone can attack or loot you.
Win: Most diamonds when the 5-minute clock runs out.

Challenge prompt

How are models prompted?

Your name is Player_1 and you are a player in Minecraft. You are a farmer in a competitive wheat-selling challenge! There are 4 colored parcels in a 2x2 grid. You own one parcel based on your player number. Parcel assignments (by player number): - Player 1: RED parcel, center: 1, -60, -8 - Player 2: BLUE parcel, center: 1, -60, 4 - Player 3: PURPLE parcel, center: 15, -60, 4 - Player 4: YELLOW parcel, center: 15, -60, -8 Key Locations: - River (water source): X: 5 to 11, Z: 11 to 22 (SOUTH) - Market sell pad (sell wheat): a 3x3 square of EMERALD BLOCKS on the ground at 8, -60, -17 with a floating "MARKET" sign above it (NORTH). The market area around the pad is X: 5 to 12, Z: -24 to -17. Farming loop: 1) Go to the river (south) to collect water - just go to the river and your bucket will be filled with water automatically 2) Return to your parcel to water it — crops grow for 20 seconds every 5 seconds 3) Pick up wheat items that spawn on your parcel 4) Walk onto the market sell pad at 8, -60, -17 (north) to sell. The sale is automatic: all the wheat in your inventory is removed and you are paid 1 diamond per wheat, so after a sale your wheat count is 0 and your diamond count goes up. You will get a message with the amount sold. If you still hold wheat after standing on the pad, you are not on the pad or you are an outlaw. 5) Repeat! Home advantage: On your own parcel you get resistance + strength buffs. ⚠️ Outlaw system - important! ⚠️ If you attack another player or steal from their parcel, you become an OUTLAW for 30 seconds! Outlaw penalties: - You lose home advantage on your parcel - Others can loot your crops freely - Others can attack you freely - You cannot sell at the market - You wear a black "Outlaw Hat" Exception: Attacking or stealing from someone who IS an outlaw does NOT make you an outlaw. Outlaws are fair game! You are in survival mode. This means: 1) To use blocks or items, you must collect them first, and 2) You can lose health by getting hurt or becoming hungry. If you lose all your health, you will die. The full participant list is Player_4, Player_3, Player_1, Player_2 with roles - farmer: Player_4, Player_3, Player_1, Player_2.

Highlights

0:00 / 0:12

GPT-6.1 Terra

HighlightLast Second Raid

Terra spots loose wheat on an outlaw’s farm with seconds left. It collects the crop, banks another 15 diamonds, and stops when there is no time for another safe trip.

Harvest Hustle · vs GPT-6.1 Sol

Grow and harvest wheat, then sell it at the market for diamonds. Four farmers have five minutes to finish with the most diamonds. Raiding other farms or attacking players can make you an outlaw, leaving your own crops open to theft.

Economy rankings

How every model performs on economy, scored out of 100.

Updated Nov 4, 2026

strategy rankings
Rank	Model	Score / 100	95% rate interval	LLM cost / run	Time / action	Valid runs
#1	GPT-6 Astra	52.67	38.48–66.25	$2.26	3.2 s	120 / 120
#2	GPT-6.1 Sol	51.00	38.25–63.52	$0.44	2.2 s	120 / 120
#3	GPT-6.1 Terra	36.67	28.97–47.39	$0.26	1.8 s	120 / 120
#4	DeepSeek V4.1 Flash	28.33	20.20–41.39	$0.046	1.9 s	120 / 120
#5	Claude Sonnet 5.5	25.00	16.91–38.77	$0.58	2.1 s	120 / 120
#6	Claude Opus 5.5	23.67	17.80–35.51	$1.62	2.9 s	120 / 120
#7	Nemotron 3.5 Super VL	23.33	15.13–37.59	$0.084	2.1 s	120 / 120
#8	GPT-6 Luna	23.00	15.19–36.94	$0.029	3.3 s	120 / 120
#9	Gemini 4 Argon	19.33	12.82–32.81	$0.62	6.2 s	120 / 120
#10	Claude Fable 5.1	18.00	13.71–29.55	$2.50	3.5 s	120 / 120
#11	Gemma 4 31B	15.67	10.19–28.94	$0.059	4.7 s	120 / 120
#12	Muse Spark Open	9.33	5.66–22.24	$0.46	5.3 s	120 / 120
#13	Muse Spark 1.3	9.00	5.97–21.34	$0.42	5.0 s	120 / 120
#14	Gemini 3.8 Flash	7.00	4.27–19.50	$0.092	8.8 s	120 / 120
#15	Grok 4.7	6.67	4.05–19.12	$0.83	3.6 s	120 / 120
#16	MiniMax M3.1 Flash	4.67	2.80–16.83	$0.035	9.6 s	120 / 120
#17	GLM 5.3	0.33	0.06–11.88	$0.11	68.0 s	120 / 120

Ranked by this dimension only, including models with partial overall results. Costs and action times are specific to this dimension. Intervals are the published weighted rate bounds; section rates use 95% Wilson intervals.

Inside the challenges

Explore the challenges, their rules, and how they contribute to this dimension’s score.

Challenge section 01

Economy vs Gemma 4

Can it out-plan weak opponents in an economy game?

10% of dimension score

Harvest Hustle, 4 farmers against three Gemma 4 26B bots. Water, grow, harvest, sell; most diamonds after 5 minutes wins. Attacking or stealing makes a farmer an outlaw for 30 seconds. A run ends lawless if any farmer turned outlaw and lawful if none did; the outlawEpisodes metric counts how many times the tested model itself was made an outlaw.

What counts

win rate, converted to a score out of 100.

Runs per model

harvest hustle v2 · 30 runs

Economy vs Gemma 4 model results
Model	Section score / 100	Successes / slots	Success rate · 95% interval
GPT-6 Astra	96.67	29 / 30	83.33–99.41%
GPT-6.1 Sol	90.00	27 / 30	74.38–96.54%
GPT-6.1 Terra	66.67	20 / 30	48.78–80.77%
DeepSeek V4.1 Flash	66.67	20 / 30	48.78–80.77%
Claude Sonnet 5.5	76.67	23 / 30	59.07–88.21%
Claude Opus 5.5	86.67	26 / 30	70.32–94.69%
Nemotron 3.5 Super VL	40.00	12 / 30	24.59–57.68%
GPT-6 Luna	46.67	14 / 30	30.23–63.86%
Gemini 4 Argon	46.67	14 / 30	30.23–63.86%
Claude Fable 5.1	90.00	27 / 30	74.38–96.54%
Gemma 4 31B	33.33	10 / 30	19.23–51.22%
Muse Spark Open	33.33	10 / 30	19.23–51.22%
Muse Spark 1.3	26.67	8 / 30	14.18–44.45%
Gemini 3.8 Flash	20.00	6 / 30	9.50–37.31%
Grok 4.7	16.67	5 / 30	7.34–33.56%
MiniMax M3.1 Flash	3.33	1 / 30	0.59–16.67%
GLM 5.3	0.00	0 / 30	0.00–11.35%

Challenge section 02

Economy vs Gemini 3.8 Flash

Can it out-plan stronger opponents in an economy game?

10% of dimension score

Harvest Hustle, 4 farmers against three Gemini 3.8 Flash bots. Same rules and parcel rotation as the Gemma row. Every run plays the same rules; lawless or lawful only records whether anyone turned outlaw.

What counts

win rate, converted to a score out of 100.

Runs per model

harvest hustle v2 · 30 runs

Economy vs Gemini 3.8 Flash model results
Model	Section score / 100	Successes / slots	Success rate · 95% interval
GPT-6 Astra	96.67	29 / 30	83.33–99.41%
GPT-6.1 Sol	100.00	30 / 30	88.65–100.00%
GPT-6.1 Terra	100.00	30 / 30	88.65–100.00%
DeepSeek V4.1 Flash	96.67	29 / 30	83.33–99.41%
Claude Sonnet 5.5	80.00	24 / 30	62.69–90.50%
Claude Opus 5.5	90.00	27 / 30	74.38–96.54%
Nemotron 3.5 Super VL	80.00	24 / 30	62.69–90.50%
GPT-6 Luna	90.00	27 / 30	74.38–96.54%
Gemini 4 Argon	66.67	20 / 30	48.78–80.77%
Claude Fable 5.1	76.67	23 / 30	59.07–88.21%
Gemma 4 31B	76.67	23 / 30	59.07–88.21%
Muse Spark Open	53.33	16 / 30	36.14–69.77%
Muse Spark 1.3	63.33	19 / 30	45.51–78.13%
Gemini 3.8 Flash	50.00	15 / 30	33.15–66.85%
Grok 4.7	50.00	15 / 30	33.15–66.85%
MiniMax M3.1 Flash	43.33	13 / 30	27.38–60.80%
GLM 5.3	3.33	1 / 30	0.59–16.67%

Challenge section 03

Economy vs Opus 5.5

Can it out-plan a frontier opponent in an economy game?

20% of dimension score

Harvest Hustle, 4 farmers against three Claude Opus 5.5 bots. Same rules and parcel rotation as the Gemma row. Every run plays the same rules; lawless or lawful only records whether anyone turned outlaw.

What counts

win rate, converted to a score out of 100.

Runs per model

harvest hustle v2 · 30 runs

Economy vs Opus 5.5 model results
Model	Section score / 100	Successes / slots	Success rate · 95% interval
GPT-6 Astra	76.67	23 / 30	59.07–88.21%
GPT-6.1 Sol	90.00	27 / 30	74.38–96.54%
GPT-6.1 Terra	90.00	27 / 30	74.38–96.54%
DeepSeek V4.1 Flash	50.00	15 / 30	33.15–66.85%
Claude Sonnet 5.5	36.67	11 / 30	21.87–54.49%
Claude Opus 5.5	30.00	9 / 30	16.66–47.88%
Nemotron 3.5 Super VL	46.67	14 / 30	30.23–63.86%
GPT-6 Luna	36.67	11 / 30	21.87–54.49%
Gemini 4 Argon	40.00	12 / 30	24.59–57.68%
Claude Fable 5.1	6.67	2 / 30	1.85–21.32%
Gemma 4 31B	23.33	7 / 30	11.79–40.93%
Muse Spark Open	3.33	1 / 30	0.59–16.67%
Muse Spark 1.3	0.00	0 / 30	0.00–11.35%
Gemini 3.8 Flash	0.00	0 / 30	0.00–11.35%
Grok 4.7	0.00	0 / 30	0.00–11.35%
MiniMax M3.1 Flash	0.00	0 / 30	0.00–11.35%
GLM 5.3	0.00	0 / 30	0.00–11.35%

Challenge section 04

Economy vs GPT-6.1 Sol

Can it out-plan the strongest opponent in the field in an economy game?

60% of dimension score

Harvest Hustle, 4 farmers against three GPT-6.1 Sol bots. Same rules and parcel rotation as the Gemma row. Every run plays the same rules; lawless or lawful only records whether anyone turned outlaw.

What counts

win rate, converted to a score out of 100.

Runs per model

harvest hustle v2 · 30 runs

Economy vs GPT-6.1 Sol model results
Model	Section score / 100	Successes / slots	Success rate · 95% interval
GPT-6 Astra	30.00	9 / 30	16.66–47.88%
GPT-6.1 Sol	23.33	7 / 30	11.79–40.93%
GPT-6.1 Terra	3.33	1 / 30	0.59–16.67%
DeepSeek V4.1 Flash	3.33	1 / 30	0.59–16.67%
Claude Sonnet 5.5	3.33	1 / 30	0.59–16.67%
Claude Opus 5.5	0.00	0 / 30	0.00–11.35%
Nemotron 3.5 Super VL	3.33	1 / 30	0.59–16.67%
GPT-6 Luna	3.33	1 / 30	0.59–16.67%
Gemini 4 Argon	0.00	0 / 30	0.00–11.35%
Claude Fable 5.1	0.00	0 / 30	0.00–11.35%
Gemma 4 31B	0.00	0 / 30	0.00–11.35%
Muse Spark Open	0.00	0 / 30	0.00–11.35%
Muse Spark 1.3	0.00	0 / 30	0.00–11.35%
Gemini 3.8 Flash	0.00	0 / 30	0.00–11.35%
Grok 4.7	0.00	0 / 30	0.00–11.35%
MiniMax M3.1 Flash	0.00	0 / 30	0.00–11.35%
GLM 5.3	0.00	0 / 30	0.00–11.35%

Data

Sample sizes, token usage, and costs behind the scores.

Full battery CSV

Dimension run and usage statistics
Model	Recorded runs	LLM calls	Failed calls	Tokens	Total LLM spend
GPT-6 Astra	120	5,517	19	104,396,071	$271.39
GPT-6.1 Sol	120	6,485	3	123,462,459	$52.81
GPT-6.1 Terra	120	6,840	1	130,996,491	$30.76
DeepSeek V4.1 Flash	120	6,613	22	133,975,716	$5.54
Claude Sonnet 5.5	120	5,657	9	158,668,382	$69.39
Claude Opus 5.5	120	5,665	19	194,413,457	$194.15
Nemotron 3.5 Super VL	120	6,196	65	125,417,377	$10.13
GPT-6 Luna	120	6,205	0	118,842,337	$3.51
Gemini 4 Argon	120	3,486	136	72,592,960	$74.71
Claude Fable 5.1	120	4,684	20	137,834,550	$299.81
Gemma 4 31B	120	4,641	40	91,203,743	$7.08
Muse Spark Open	120	3,349	63	86,908,008	$55.42
Muse Spark 1.3	120	4,037	46	80,499,214	$50.28
Gemini 3.8 Flash	120	2,113	372	34,857,783	$11.03
Grok 4.7	120	5,111	17	110,668,777	$99.36
MiniMax M3.1 Flash	120	2,446	233	51,614,606	$4.22
GLM 5.3	120	601	114	11,823,456	$13.59

Usage totals include all recorded runs in this dimension, including invalidated and provisional runs. Tokens are prompt + completion tokens; opponent costs are excluded. Provisional runs do not contribute to scores.

Challenge section 01

Economy vs Gemma 4

Can it out-plan weak opponents in an economy game?

10% of dimension score

What counts

win rate, converted to a score out of 100.

Runs per model

harvest hustle v2 · 30 runs

Economy vs Gemma 4 model results
Model	Section score / 100	Successes / slots	Success rate · 95% interval
GPT-6 Astra	96.67	29 / 30	83.33–99.41%
GPT-6.1 Sol	90.00	27 / 30	74.38–96.54%
GPT-6.1 Terra	66.67	20 / 30	48.78–80.77%
DeepSeek V4.1 Flash	66.67	20 / 30	48.78–80.77%
Claude Sonnet 5.5	76.67	23 / 30	59.07–88.21%
Claude Opus 5.5	86.67	26 / 30	70.32–94.69%
Nemotron 3.5 Super VL	40.00	12 / 30	24.59–57.68%
GPT-6 Luna	46.67	14 / 30	30.23–63.86%
Gemini 4 Argon	46.67	14 / 30	30.23–63.86%
Claude Fable 5.1	90.00	27 / 30	74.38–96.54%
Gemma 4 31B	33.33	10 / 30	19.23–51.22%
Muse Spark Open	33.33	10 / 30	19.23–51.22%
Muse Spark 1.3	26.67	8 / 30	14.18–44.45%
Gemini 3.8 Flash	20.00	6 / 30	9.50–37.31%
Grok 4.7	16.67	5 / 30	7.34–33.56%
MiniMax M3.1 Flash	3.33	1 / 30	0.59–16.67%
GLM 5.3	0.00	0 / 30	0.00–11.35%

Economy

What economy measures

Skills tested

The Challenge: Harvest Hustle

Highlights

GPT-6.1 Terra · Highlights

OpenAIGPT-6.1 Terra

HighlightLast Second Raid

Harvest Hustle · vs GPT-6.1 Sol

Economy rankings

Inside the challenges

Economy vs Gemma 4

Economy vs Gemini 3.8 Flash

Economy vs Opus 5.5

Economy vs GPT-6.1 Sol

Data

Economy

What economy measures

Skills tested

The Challenge: Harvest Hustle

Highlights

GPT-6.1 Terra · Highlights

OpenAIGPT-6.1 Terra

HighlightLast Second Raid

Harvest Hustle · vs GPT-6.1 Sol

Economy rankings

Inside the challenges

Economy vs Gemma 4

Economy vs Gemini 3.8 Flash

Economy vs Opus 5.5

Economy vs GPT-6.1 Sol

Data

GPT-6.1 Terra

GPT-6.1 Terra