
0:00 / 3:04
Dimension 01 / 05 · Up against the environment
Stay alive when the world turns against you.
Two challenges, no respawn: a zombie graveyard and a full night in the open, with no hiding allowed.

17
Models scored
With published dimension scores
60
Scored slots per model
Excludes provisional runs
1,020
Recorded runs
Across models in this dimension
20%
Of the overall score
Five equally weighted dimensions
A solo stand on a graveyard world. The zombies arrive at 90 seconds, and the run lasts 3 minutes.
A locked 10-minute night on hard difficulty, in an untouched river world. Hostile mobs keep spawning on the surface around the player until the night ends.
Most benchmarks hand a model a problem and wait for the answer. Deployed agents don’t get that: the world keeps moving, threats arrive on their own schedule, and resources run down whether the agent acts or not. Survival measures whether a model notices danger it didn’t cause, spends health and supplies carefully, and commits to a plan in time. No respawn, no hiding: a model that hesitates or ignores the threat doesn’t make it to morning.

Survival · Zombie Apocalypse · Survived 30 of 30
Opus built a compact cobblestone enclosure and then ran a loop that checked and repaired its walls and roof. It remained protected through the zombie wave and survived without recorded health loss.
1 of 3: Zombie Apocalypse, Claude Opus 5.5.
How every model performs on survival, scored out of 100.
Updated
| Rank | Model | Score / 100 | 95% rate interval | LLM cost / run | Time / action | Valid runs |
|---|---|---|---|---|---|---|
| #1 | Claude Opus 5.5 | 61.00 | 44.14–75.36 | $0.98 | 3.3 s | 60 / 60 |
| #2 | GPT-6 Astra | 58.00 | 41.39–72.79 | $1.49 | 3.0 s | 60 / 60 |
| #2 | GPT-6.1 Terra | 58.00 | 41.39–72.79 | $0.13 | 1.8 s | 60 / 60 |
| #4 | GPT-6.1 Sol | 52.00 | 36.07–67.47 | $0.23 | 2.7 s | 60 / 60 |
| #5 | Claude Sonnet 5.5 | 46.00 | 31.00–61.91 | $0.45 | 2.3 s | 60 / 60 |
| #6 | Claude Fable 5.1 | 39.66 | 25.64–56.04 | $1.60 | 3.2 s | 60 / 60 |
| #7 | DeepSeek V4.1 Flash | 39.00 | 24.75–55.75 | $0.024 | 2.1 s | 60 / 60 |
| #8 | Muse Spark 1.3 | 36.67 | 23.33–53.03 | $0.77 | 2.5 s | 60 / 60 |
| #9 | Grok 4.7 | 34.33 | 20.55–51.67 | $0.43 | 3.8 s | 60 / 60 |
| #10 | Gemini 4 Argon | 31.00 | 19.48–46.84 | $0.37 | 5.3 s | 60 / 60 |
| #11 | Gemini 3.8 Flash | 24.67 | 14.94–40.15 | $0.11 | 4.7 s | 60 / 60 |
| #12 | MiniMax M3.1 Flash | 18.00 | 8.27–35.00 | $0.024 | 2.8 s | 60 / 60 |
| #13 | Gemma 4 31B | 17.66 | 8.70–33.98 | $0.047 | 3.7 s | 60 / 60 |
| #14 | GLM 5.3 | 16.00 | 7.24–32.48 | $0.10 | 6.1 s | 60 / 60 |
| #15 | GPT-6 Luna | 13.67 | 7.57–28.01 | $0.014 | 4.0 s | 60 / 60 |
| #16 | Nemotron 3.5 Super VL | 13.33 | 5.85–29.14 | $0.024 | 3.2 s | 60 / 60 |
| #17 | Muse Spark Open | 9.67 | 8.33–20.16 | $0.35 | 3.2 s | 60 / 60 |
Ranked by this dimension only, including models with partial overall results. Costs and action times are specific to this dimension. Intervals are the published weighted rate bounds; section rates use 95% Wilson intervals.
Explore the challenges, their rules, and how they contribute to this dimension’s score.
Challenge section 01
Can it stay alive under pressure?
Zombie Apocalypse, solo, on the Graveyard world. Zombies arrive at 90 seconds; be alive at the 3-minute mark.
What counts
survival rate, converted to a score out of 100.
Runs per model
| Model | Section score / 100 | Successes / slots | Success rate · 95% interval |
|---|---|---|---|
| Claude Opus 5.5 | 100.00 | 30 / 30 | 88.65–100.00% |
| GPT-6 Astra | 100.00 | 30 / 30 | 88.65–100.00% |
| GPT-6.1 Terra | 100.00 | 30 / 30 | 88.65–100.00% |
| GPT-6.1 Sol | 100.00 | 30 / 30 | 88.65–100.00% |
| Claude Sonnet 5.5 | 100.00 | 30 / 30 | 88.65–100.00% |
| Claude Fable 5.1 | 96.67 | 29 / 30 | 83.33–99.41% |
| DeepSeek V4.1 Flash | 90.00 | 27 / 30 | 74.38–96.54% |
| Muse Spark 1.3 | 96.67 | 29 / 30 | 83.33–99.41% |
| Grok 4.7 | 73.33 | 22 / 30 | 55.55–85.82% |
| Gemini 4 Argon | 100.00 | 30 / 30 | 88.65–100.00% |
| Gemini 3.8 Flash | 96.67 | 29 / 30 | 83.33–99.41% |
| MiniMax M3.1 Flash | 30.00 | 9 / 30 | 16.66–47.88% |
| Gemma 4 31B | 56.67 | 17 / 30 | 39.20–72.62% |
| GLM 5.3 | 40.00 | 12 / 30 | 24.59–57.68% |
| GPT-6 Luna | 76.67 | 23 / 30 | 59.07–88.21% |
| Nemotron 3.5 Super VL | 43.33 | 13 / 30 | 27.38–60.80% |
| Muse Spark Open | 96.67 | 29 / 30 | 83.33–99.41% |
Challenge section 02
Can it survive a hard night in the open?
Nightfall, solo, survival mode on hard difficulty: a locked 10-minute night without hiding (medium), plus a hunt of 2 zombies and 1 skeleton (hard). The no-hiding rule is judged on every surviving run.
What counts
nights survived, converted to a score out of 100.
Judging applies to winning runs, with 3 samples and majority aggregation.
Runs per model
---
id: nightfall-no-hiding
version: v2
kind: rule
aggregateField: hid
---
# Nightfall: the no-hiding rule
An AI agent ("the survivor") was asked to survive a ten-minute Minecraft night on hard difficulty
without hiding. It survived the night, so the only question is whether it respected the rule as the
challenge stated it to the survivor. The challenge's own words were: digging down to hide below the
surface is forbidden; sealing yourself inside a fully enclosed pocket of blocks (a hole capped over
your head, a tunnel, a closed box) is forbidden; fighting, running, climbing and building are allowed;
"a wall or a roofless fort is fine; a sealed coffin is not."
Hiding therefore means exactly one of these three things:
- a sealed enclosure: blocks on every side AND overhead, of any material, placed or dug, so that no
mob could reach the survivor from any direction;
- a hole: digging down below the starting ground level and staying there, whether or not it was
capped;
- a tunnel: digging sideways into terrain and staying inside.
Everything else is allowed, however passive it looks. In particular, these are NOT hiding:
- standing on a pillar, column, tower or platform of any height, with or without a wall, rim or
parapet around its edge, as long as nothing was placed over the survivor's head;
- a roofless fort or wall at ground level;
- sheltering under trees, standing in water, climbing terrain;
- taking no damage, dealing no damage, killing nothing and never moving. A survivor on a legal tower
will usually show all four, and the rule does not require it to fight.
Evidence you are given, in the facts and texts:
- cover ticks (ticks spent with a block directly overhead), minimum y against starting y, maximum y,
blocks broken and placed, damage taken, kills, distance from the start;
- the survivor's chat and the code it ran, filtered to digging, placing and movement.
How to read it:
- Going UP (max_y above start_y with blocks placed) is building, which is allowed. Going DOWN (min_y
below start_y with blocks broken) is digging, which is hiding if the survivor stayed down there.
- Cover ticks come from a roof the survivor placed over itself, from a tree canopy, or from an
overhang it walked under. Only a roof it placed over its own position, together with walls, makes
a sealed enclosure. Cover ticks of zero rule out a sealed enclosure entirely.
- Rings or walls placed around the survivor's own position are a parapet, not a seal, unless blocks
were also placed above it.
- Read the code for what was actually placed and where, relative to the survivor's position, rather
than inferring intent from the counts alone.
Worked examples:
- A survivor that jump-places ten blocks under itself, adds two rings of blocks around the top of
that column, then waits all night: max_y ten above start, cover ticks zero, no blocks broken, no
damage, no kills. **Not hiding.** This is the roofless tower the challenge allows.
- A survivor with 10 kills, 122 damage taken, 555 cover ticks from water under trees, 2 blocks placed,
no enclosure and no digging. **Not hiding.**
- A survivor that digs two blocks down, places a block over the hole and waits: min_y below start,
blocks broken, cover ticks for most of the night. **Hiding, dug_in_hole.**
- A survivor that builds walls around itself at ground level and a roof on top, cover ticks for
most of the night and no damage. **Hiding, sealed_box.**
- A survivor that tunnels sideways into a hill and waits. **Hiding, tunnel.**
Answer with `hid` (true or false), the `hiding_type` (`none`, `sealed_box`, `dug_in_hole`, `tunnel`,
`other`), the specific pieces of evidence you relied on, a short rationale, and your confidence.
`other` is for a genuine enclosure that fits none of the three names, never for a tower, platform,
wall or passive play.
| Model | Section score / 100 | Successes / slots | Success rate · 95% interval |
|---|---|---|---|
| Claude Opus 5.5 | 56.67 | 17 / 30 Locked night: 8/15 Night hunt: 9/15 | 39.20–72.62% |
| GPT-6 Astra | 53.33 | 16 / 30 Locked night: 6/15 Night hunt: 10/15 | 36.14–69.77% |
| GPT-6.1 Terra | 53.33 | 16 / 30 Locked night: 7/15 Night hunt: 9/15 | 36.14–69.77% |
| GPT-6.1 Sol | 46.67 | 14 / 30 Locked night: 4/15 Night hunt: 10/15 | 30.23–63.86% |
| Claude Sonnet 5.5 | 40.00 | 12 / 30 Locked night: 6/15 Night hunt: 6/15 | 24.59–57.68% |
| Claude Fable 5.1 | 33.33 | 10 / 30 Locked night: 4/15 Night hunt: 6/15 | 19.23–51.22% |
| DeepSeek V4.1 Flash | 33.33 | 10 / 30 Locked night: 2/15 Night hunt: 8/15 | 19.23–51.22% |
| Muse Spark 1.3 | 30.00 | 9 / 30 Locked night: 3/15 Night hunt: 6/15 | 16.66–47.88% |
| Grok 4.7 | 30.00 | 9 / 30 Locked night: 2/15 Night hunt: 7/15 | 16.66–47.88% |
| Gemini 4 Argon | 23.33 | 7 / 30 Locked night: 1/15 Night hunt: 6/15 | 11.79–40.93% |
| Gemini 3.8 Flash | 16.67 | 5 / 30 Locked night: 1/15 Night hunt: 4/15 | 7.34–33.56% |
| MiniMax M3.1 Flash | 16.67 | 5 / 30 Locked night: 2/15 Night hunt: 3/15 | 7.34–33.56% |
| Gemma 4 31B | 13.33 | 4 / 30 Locked night: 3/15 Night hunt: 1/15 | 5.31–29.68% |
| GLM 5.3 | 13.33 | 4 / 30 Locked night: 1/15 Night hunt: 3/15 | 5.31–29.68% |
| GPT-6 Luna | 6.67 | 2 / 30 Locked night: 2/15 Night hunt: 0/15 | 1.85–21.32% |
| Nemotron 3.5 Super VL | 10.00 | 3 / 30 Locked night: 2/15 Night hunt: 1/15 | 3.46–25.62% |
| Muse Spark Open | 0.00 | 0 / 30 Locked night: 0/15 Night hunt: 0/15 | 0.00–11.35% |
Sample sizes, token usage, costs, and challenge telemetry behind the scores.
| Model | Recorded runs | Invalidated | Provisional | LLM calls | Failed calls | Tokens | Total LLM spend |
|---|---|---|---|---|---|---|---|
| Claude Opus 5.5 | 60 | 0 | 0 | 2,504 | 0 | 70,913,661 | $58.64 |
| GPT-6 Astra | 60 | 0 | 0 | 2,292 | 0 | 40,934,487 | $89.42 |
| GPT-6.1 Terra | 60 | 0 | 0 | 2,680 | 0 | 46,713,031 | $8.02 |
| GPT-6.1 Sol | 60 | 0 | 0 | 2,223 | 0 | 39,719,279 | $13.92 |
| Claude Sonnet 5.5 | 60 | 0 | 0 | 2,612 | 0 | 68,759,924 | $26.79 |
| Claude Fable 5.1 | 60 | 0 | 0 | 1,771 | 0 | 49,310,888 | $96.25 |
| DeepSeek V4.1 Flash | 60 | 0 | 0 | 1,610 | 0 | 30,568,713 | $1.45 |
| Muse Spark 1.3 | 60 | 0 | 0 | 2,216 | 0 | 67,121,898 | $46.20 |
| Grok 4.7 | 60 | 0 | 0 | 1,208 | 0 | 28,461,595 | $25.77 |
| Gemini 4 Argon | 60 | 0 | 0 | 1,327 | 0 | 27,170,442 | $22.49 |
| Gemini 3.8 Flash | 60 | 0 | 0 | 1,315 | 0 | 23,765,979 | $6.62 |
| MiniMax M3.1 Flash | 60 | 0 | 0 | 1,373 | 0 | 31,688,891 | $1.46 |
| Gemma 4 31B | 60 | 0 | 0 | 1,716 | 0 | 29,996,551 | $2.80 |
| GLM 5.3 | 60 | 0 | 0 | 663 | 0 | 12,988,767 | $6.19 |
| GPT-6 Luna | 60 | 0 | 0 | 1,734 | 0 | 31,447,447 | $0.86 |
| Nemotron 3.5 Super VL | 60 | 0 | 0 | 945 | 0 | 16,658,654 | $1.44 |
| Muse Spark Open | 60 | 0 | 0 | 1,267 | 0 | 36,243,730 | $21.17 |
Usage totals include all recorded runs in this dimension, including invalidated and provisional runs. Tokens are prompt + completion tokens; opponent costs are excluded. Provisional runs do not contribute to scores.
Section results, sample counts, and the extra metrics recorded by each challenge.
100.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
56.67 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
Section results, sample counts, and the extra metrics recorded by each challenge.
100.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
53.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
Section results, sample counts, and the extra metrics recorded by each challenge.
100.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
53.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
Section results, sample counts, and the extra metrics recorded by each challenge.
100.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
46.67 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
Section results, sample counts, and the extra metrics recorded by each challenge.
100.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
40.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
Section results, sample counts, and the extra metrics recorded by each challenge.
96.67 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
33.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
Section results, sample counts, and the extra metrics recorded by each challenge.
90.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
33.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
Section results, sample counts, and the extra metrics recorded by each challenge.
96.67 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
30.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
Section results, sample counts, and the extra metrics recorded by each challenge.
73.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
30.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
Section results, sample counts, and the extra metrics recorded by each challenge.
100.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
23.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
Section results, sample counts, and the extra metrics recorded by each challenge.
96.67 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
16.67 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
Section results, sample counts, and the extra metrics recorded by each challenge.
30.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
16.67 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
Section results, sample counts, and the extra metrics recorded by each challenge.
56.67 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
13.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
Section results, sample counts, and the extra metrics recorded by each challenge.
40.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
13.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
Section results, sample counts, and the extra metrics recorded by each challenge.
76.67 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
6.67 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
Section results, sample counts, and the extra metrics recorded by each challenge.
43.33 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
10.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
Section results, sample counts, and the extra metrics recorded by each challenge.
96.67 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment
0.00 / 100 · 30 scored runs · 30 planned slots · 0 awaiting judgment