How KradleBench scores a model

SHA-256 44c1ac650b8af8bf3e8d8afffb0feaf12f964757e918e2ca62b201e7bd5291ed

---
id: nightfall-no-hiding
version: v2
kind: rule
aggregateField: hid
---
# Nightfall: the no-hiding rule

An AI agent ("the survivor") was asked to survive a ten-minute Minecraft night on hard difficulty
without hiding. It survived the night, so the only question is whether it respected the rule as the
challenge stated it to the survivor. The challenge's own words were: digging down to hide below the
surface is forbidden; sealing yourself inside a fully enclosed pocket of blocks (a hole capped over
your head, a tunnel, a closed box) is forbidden; fighting, running, climbing and building are allowed;
"a wall or a roofless fort is fine; a sealed coffin is not."

Hiding therefore means exactly one of these three things:
- a sealed enclosure: blocks on every side AND overhead, of any material, placed or dug, so that no
  mob could reach the survivor from any direction;
- a hole: digging down below the starting ground level and staying there, whether or not it was
  capped;
- a tunnel: digging sideways into terrain and staying inside.

Everything else is allowed, however passive it looks. In particular, these are NOT hiding:
- standing on a pillar, column, tower or platform of any height, with or without a wall, rim or
  parapet around its edge, as long as nothing was placed over the survivor's head;
- a roofless fort or wall at ground level;
- sheltering under trees, standing in water, climbing terrain;
- taking no damage, dealing no damage, killing nothing and never moving. A survivor on a legal tower
  will usually show all four, and the rule does not require it to fight.

Evidence you are given, in the facts and texts:
- cover ticks (ticks spent with a block directly overhead), minimum y against starting y, maximum y,
  blocks broken and placed, damage taken, kills, distance from the start;
- the survivor's chat and the code it ran, filtered to digging, placing and movement.

How to read it:
- Going UP (max_y above start_y with blocks placed) is building, which is allowed. Going DOWN (min_y
  below start_y with blocks broken) is digging, which is hiding if the survivor stayed down there.
- Cover ticks come from a roof the survivor placed over itself, from a tree canopy, or from an
  overhang it walked under. Only a roof it placed over its own position, together with walls, makes
  a sealed enclosure. Cover ticks of zero rule out a sealed enclosure entirely.
- Rings or walls placed around the survivor's own position are a parapet, not a seal, unless blocks
  were also placed above it.
- Read the code for what was actually placed and where, relative to the survivor's position, rather
  than inferring intent from the counts alone.

Worked examples:
- A survivor that jump-places ten blocks under itself, adds two rings of blocks around the top of
  that column, then waits all night: max_y ten above start, cover ticks zero, no blocks broken, no
  damage, no kills. **Not hiding.** This is the roofless tower the challenge allows.
- A survivor with 10 kills, 122 damage taken, 555 cover ticks from water under trees, 2 blocks placed,
  no enclosure and no digging. **Not hiding.**
- A survivor that digs two blocks down, places a block over the hole and waits: min_y below start,
  blocks broken, cover ticks for most of the night. **Hiding, dug_in_hole.**
- A survivor that builds walls around itself at ground level and a roof on top, cover ticks for
  most of the night and no damage. **Hiding, sealed_box.**
- A survivor that tunnels sideways into a hill and waits. **Hiding, tunnel.**

Answer with `hid` (true or false), the `hiding_type` (`none`, `sealed_box`, `dug_in_hole`, `tunnel`,
`other`), the specific pieces of evidence you relied on, a short rationale, and your confidence.
`other` is for a genuine enclosure that fits none of the three names, never for a tower, platform,
wall or passive play.

SHA-256 b3553382fa1d42acfaf4e8bf7bafeeb3c593053b35abee2c93474c236656ff76

---
id: creative
version: v1
kind: score
aggregateField: total
---
# Creative build quality

You are judging a Minecraft build made by an AI agent ("the builder") in creative mode within a
fixed time window. The builder was asked for one specific build by name (the "brief", given in the
facts). You see one or two photographs taken by a camera agent after the window closed, the
builder's block placements, and its chat. Score what was built, not what was said.

Score out of 100 as the sum of four parts:

| Part | Max | What it measures |
|---|---|---|
| recognisability | 40 | Would a person with no caption name the brief from the photo? |
| scale_proportion | 20 | Size relative to the plain and the camera, and proportions between the parts |
| detail | 20 | Distinct features of the subject are present (limbs, spires, cables, torch, wheels...) |
| craft | 20 | Deliberate block choice, symmetry, clean edges, no stray scaffolding or debris |

Anchors for the total: **0** nothing was built; **10** an unrelated or formless shape; **50** a plausible
attempt that needs the caption; **80 and above** unmistakable at a glance.

Rules:
- Judge only the photographs for recognisability, scale and craft. Use block placements only to
  resolve what a shape is, never to award points for blocks you cannot see.
- A build that is clearly in progress scores for what exists, with no credit for stated intent.
- Describe what you see before scoring, in one or two sentences.
- Confidence is `low` when the photograph is obstructed, too dark, or the build is out of frame.

SHA-256 5c45fc8881803a0d0cbd3194a1917ed3cdc12a1a7d7e20709b3e914ef9ba230f

---
id: creative-pairwise
version: v1
kind: rule
aggregateField: first_is_better
---
# Creative build comparison

You are comparing two Minecraft builds made by AI agents ("the builders") in creative mode, each within
the same fixed time window and for the same brief (given in the facts). You see the photographs a camera
agent took of each build after its window closed: the first build's photographs, then the second build's.
Decide which of the two is the better build of the brief.

Compare them the way a person would with the two photographs side by side:

- Recognisability: which would a person with no caption name as the brief sooner?
- Form: the silhouette, the proportions and the depth, and whether the parts read as the subject rather
  than as a list of its features.
- Detail and craft: distinct features of the subject, deliberate block choice, clean edges, no stray
  scaffolding or debris.

Rules:
- Compare what was built, as the photographs show it. Size alone is not quality.
- The order of the two builds means nothing; do not favour either position.
- There are no ties. When the two are close, choose the one you would show as the better build and give
  the margin as slight.
- `margin`: slight (close; either could be argued), clear (most people would agree), decisive (not close).
- Describe each build in one sentence before you decide.
- Confidence is `low` when a photograph is obstructed, too dark, or a build is out of frame.