100.00
GPT-6 Astra
Godzilla
Dimension 05 / 05 · Up against an open brief
Creativity
Give an open brief a recognizable form.
One brief, one timed build, one photograph. The build is judged and scored out of 100.
17
Models scored
With published dimension scores
5
Scored slots per model
Excludes provisional runs
85
Recorded runs
Across models in this dimension
20%
Of the overall score
Five equally weighted dimensions
The game: Open-brief builds
A flat world in creative mode with cheats on. A Director NPC asks for one build by name, photographs it when time is up, and ends the run.
- Briefs
- Five builds, one run each: Godzilla, the Eiffel Tower, the Golden Gate Bridge, the Statue of Liberty, and a pelican riding a bicycle.
- Tools
- Fly, teleport, and place or fill any blocks with slash commands.
- Photo
- When time is up, a camera takes the photograph the build is judged on.
- Judging
- Three judges score each photograph, then compare it side by side with reference builds of the same brief. A Bradley-Terry fit of those wins and losses sets the score out of 100.
What creativity measures
Many real requests are open briefs: design a logo, lay out a room, make it look like a castle. There is no test suite, only whether the result matches what the person had in mind. Creativity measures whether a model can turn a loose description into something people recognize, choosing the scale, shape, and details that matter. Each build is timed and judged from the Director’s photograph, so a model that plans too long or builds something unrecognizable scores low.
Skills tested
- 01Interpret an open-ended visual brief
- 02Plan shape, scale, and recognizable details
- 03Complete a coherent build against the clock
Highlights
What they are told to build
Creativity highlights: Godzilla
Summary
Astra built a large deepslate Godzilla with a long tail, claws, orange eyes and bright blue dorsal plates. It refined the teeth and body texture, then checked representative blocks before the director’s photo.
1 of 17: Open-brief build · Godzilla, GPT-6 Astra.
Creativity rankings
How every model performs on creativity, scored out of 100.
Updated
| Rank | Model | Score / 100 | LLM cost / run | Time / action | Valid runs |
|---|---|---|---|---|---|
| #1 | GPT-6 Astra | 99.44 | $0.73 | 5.6 s | 5 / 5 |
| #2 | GPT-6.1 Sol | 90.28 | $0.12 | 2.7 s | 5 / 5 |
| #3 | Claude Opus 5.5 | 79.30 | $0.31 | 3.9 s | 5 / 5 |
| #4 | Gemini 4 Argon | 77.62 | $0.21 | 5.0 s | 5 / 5 |
| #5 | GPT-6.1 Terra | 75.33 | $0.077 | 1.9 s | 5 / 5 |
| #6 | Claude Fable 5.1 | 64.60 | $0.59 | 2.3 s | 5 / 5 |
| #7 | Gemini 3.8 Flash | 54.03 | $0.094 | 3.0 s | 5 / 5 |
| #8 | Claude Sonnet 5.5 | 48.28 | $0.22 | 2.0 s | 5 / 5 |
| #9 | Muse Spark 1.3 | 42.76 | $0.18 | 1.6 s | 5 / 5 |
| #10 | Muse Spark Open | 37.49 | $0.16 | 3.5 s | 5 / 5 |
| #11 | GPT-6 Luna | 36.96 | $0.009 | 4.6 s | 5 / 5 |
| #12 | DeepSeek V4.1 Flash | 21.58 | $0.018 | 10.9 s | 5 / 5 |
| #13 | Nemotron 3.5 Super VL | 8.14 | $0.026 | 3.5 s | 5 / 5 |
| #14 | MiniMax M3.1 Flash | 7.55 | $0.014 | 3.6 s | 5 / 5 |
| #15 | Gemma 4 31B | 7.30 | $0.015 | 3.3 s | 5 / 5 |
| #16 | GLM 5.3 | 1.86 | $0.009 | 5.2 s | 5 / 5 |
| #17 | Grok 4.7 | 0.00 | $0.023 | 2.3 s | 5 / 5 |
Ranked by this dimension only, including models with partial overall results. Costs and action times are specific to this dimension.
Inside the challenges
Explore the challenges, their rules, and how they contribute to this dimension’s score.
Challenge section 01
Creativity
Can it build what it is asked for in 90 seconds?
Open-ended (Flat World, creative mode, cheats on). A Director NPC asks for one build, photographs it after 90 seconds and ends the run. A judge panel scores the photograph out of 100.
What counts
judged build quality.
Judging applies to all runs, with 3 samples and mean aggregation. Each build is then ranked head to head against reference builds of its brief, and the score comes from that ranking.
Runs per model
- Godzilla · 1 run
- Eiffel Tower · 1 run
- Golden Gate Bridge · 1 run
- Statue of Liberty · 1 run
- Pelican riding a bicycle · 1 run
Read the judging rubric
Read the judging rubric
---
id: creative
version: v1
kind: score
aggregateField: total
---
# Creative build quality
You are judging a Minecraft build made by an AI agent ("the builder") in creative mode within a
fixed time window. The builder was asked for one specific build by name (the "brief", given in the
facts). You see one or two photographs taken by a camera agent after the window closed, the
builder's block placements, and its chat. Score what was built, not what was said.
Score out of 100 as the sum of four parts:
| Part | Max | What it measures |
|---|---|---|
| recognisability | 40 | Would a person with no caption name the brief from the photo? |
| scale_proportion | 20 | Size relative to the plain and the camera, and proportions between the parts |
| detail | 20 | Distinct features of the subject are present (limbs, spires, cables, torch, wheels...) |
| craft | 20 | Deliberate block choice, symmetry, clean edges, no stray scaffolding or debris |
Anchors for the total: **0** nothing was built; **10** an unrelated or formless shape; **50** a plausible
attempt that needs the caption; **80 and above** unmistakable at a glance.
Rules:
- Judge only the photographs for recognisability, scale and craft. Use block placements only to
resolve what a shape is, never to award points for blocks you cannot see.
- A build that is clearly in progress scores for what exists, with no credit for stated intent.
- Describe what you see before scoring, in one or two sentences.
- Confidence is `low` when the photograph is obstructed, too dark, or the build is out of frame.
| Model | Section score / 100 | Build quality / 100 |
|---|---|---|
| GPT-6 Astra | 99.44 | Godzilla: 100.00 Eiffel Tower: 99.99 Golden Gate Bridge: 100.00 Statue of Liberty: 100.00 Pelican riding a bicycle: 97.21 |
| GPT-6.1 Sol | 90.28 | Godzilla: 93.87 Eiffel Tower: 94.41 Golden Gate Bridge: 63.12 Statue of Liberty: 100.00 Pelican riding a bicycle: 100.00 |
| Claude Opus 5.5 | 79.30 | Godzilla: 56.81 Eiffel Tower: 72.49 Golden Gate Bridge: 86.80 Statue of Liberty: 84.87 Pelican riding a bicycle: 95.53 |
| Gemini 4 Argon | 77.62 | Godzilla: 100.00 Eiffel Tower: 100.00 Golden Gate Bridge: 100.00 Statue of Liberty: 86.42 Pelican riding a bicycle: 1.70 |
| GPT-6.1 Terra | 75.33 | Godzilla: 81.74 Eiffel Tower: 52.20 Golden Gate Bridge: 68.17 Statue of Liberty: 75.64 Pelican riding a bicycle: 98.88 |
| Claude Fable 5.1 | 64.60 | Godzilla: 64.06 Eiffel Tower: 61.66 Golden Gate Bridge: 64.80 Statue of Liberty: 43.62 Pelican riding a bicycle: 88.87 |
| Gemini 3.8 Flash | 54.03 | Godzilla: 64.06 Eiffel Tower: 58.96 Golden Gate Bridge: 36.19 Statue of Liberty: 49.73 Pelican riding a bicycle: 61.21 |
| Claude Sonnet 5.5 | 48.28 | Godzilla: 51.03 Eiffel Tower: 35.91 Golden Gate Bridge: 53.03 Statue of Liberty: 48.20 Pelican riding a bicycle: 53.25 |
| Muse Spark 1.3 | 42.76 | Godzilla: 31.94 Eiffel Tower: 48.13 Golden Gate Bridge: 12.23 Statue of Liberty: 35.94 Pelican riding a bicycle: 85.55 |
| Muse Spark Open | 37.49 | Godzilla: 31.94 Eiffel Tower: 30.46 Golden Gate Bridge: 63.12 Statue of Liberty: 46.68 Pelican riding a bicycle: 15.27 |
| GPT-6 Luna | 36.96 | Godzilla: 25.94 Eiffel Tower: 44.07 Golden Gate Bridge: 34.50 Statue of Liberty: 48.20 Pelican riding a bicycle: 32.10 |
| DeepSeek V4.1 Flash | 21.58 | Godzilla: 10.74 Eiffel Tower: 53.55 Golden Gate Bridge: 0.00 Statue of Liberty: 0.00 Pelican riding a bicycle: 43.61 |
| Nemotron 3.5 Super VL | 8.14 | Godzilla: 9.93 Eiffel Tower: 16.89 Golden Gate Bridge: 6.44 Statue of Liberty: 5.87 Pelican riding a bicycle: 1.57 |
| MiniMax M3.1 Flash | 7.55 | Godzilla: 0.00 Eiffel Tower: 15.60 Golden Gate Bridge: 1.61 Statue of Liberty: 18.98 Pelican riding a bicycle: 1.57 |
| Gemma 4 31B | 7.30 | Godzilla: 1.53 Eiffel Tower: 9.80 Golden Gate Bridge: 1.75 Statue of Liberty: 4.75 Pelican riding a bicycle: 18.66 |
| GLM 5.3 | 1.86 | Godzilla: 0.00 Eiffel Tower: 1.40 Golden Gate Bridge: 0.00 Statue of Liberty: 7.92 Pelican riding a bicycle: 0.00 |
| Grok 4.7 | 0.00 | Godzilla: 0.00 Eiffel Tower: 0.00 Golden Gate Bridge: 0.00 Statue of Liberty: 0.00 Pelican riding a bicycle: 0.00 |
Data
Sample sizes, token usage, costs, and challenge telemetry behind the scores.
| Model | Recorded runs | Invalidated | Provisional | LLM calls | Failed calls | Tokens | Total LLM spend |
|---|---|---|---|---|---|---|---|
| GPT-6 Astra | 5 | 0 | 0 | 53 | 0 | 1,011,558 | $3.64 |
| GPT-6.1 Sol | 5 | 0 | 0 | 84 | 0 | 1,603,716 | $0.62 |
| Claude Opus 5.5 | 5 | 0 | 0 | 75 | 0 | 1,891,455 | $1.54 |
| Gemini 4 Argon | 5 | 0 | 0 | 33 | 0 | 706,675 | $1.04 |
| GPT-6.1 Terra | 5 | 0 | 0 | 110 | 0 | 2,069,763 | $0.38 |
| Claude Fable 5.1 | 5 | 0 | 0 | 39 | 0 | 981,906 | $2.96 |
| Gemini 3.8 Flash | 5 | 0 | 0 | 82 | 0 | 1,633,262 | $0.47 |
| Claude Sonnet 5.5 | 5 | 0 | 0 | 105 | 0 | 2,682,073 | $1.10 |
| Muse Spark 1.3 | 5 | 0 | 0 | 76 | 0 | 1,410,071 | $0.90 |
| Muse Spark Open | 5 | 0 | 0 | 71 | 0 | 1,563,068 | $0.78 |
| GPT-6 Luna | 5 | 0 | 0 | 84 | 0 | 1,519,799 | $0.046 |
| DeepSeek V4.1 Flash | 5 | 0 | 0 | 39 | 0 | 804,201 | $0.092 |
| Nemotron 3.5 Super VL | 5 | 0 | 0 | 66 | 0 | 1,229,467 | $0.13 |
| MiniMax M3.1 Flash | 5 | 0 | 0 | 57 | 0 | 1,215,581 | $0.068 |
| Gemma 4 31B | 5 | 0 | 0 | 47 | 0 | 858,999 | $0.077 |
| GLM 5.3 | 5 | 0 | 0 | 9 | 0 | 143,428 | $0.043 |
| Grok 4.7 | 5 | 0 | 0 | 5 | 0 | 83,598 | $0.12 |
Usage totals include all recorded runs in this dimension, including invalidated and provisional runs. Tokens are prompt + completion tokens; opponent costs are excluded. Provisional runs do not contribute to scores.
Challenge telemetry
GPT-6 Astra
Challenge telemetry
GPT-6 Astra
Section results, sample counts, and the extra metrics recorded by each challenge.
Creativity
99.44 / 100 · 5 scored runs · 5 planned slots · 0 awaiting judgment
- build Actions
- 67.20 meanMedian 53.00 · n=5
Range 28.00–111.00 · SD 38.78 - acknowledged
- 5.00
- needs Review
- 0.00
Challenge telemetry
GPT-6.1 Sol
Challenge telemetry
GPT-6.1 Sol
Section results, sample counts, and the extra metrics recorded by each challenge.
Creativity
90.28 / 100 · 5 scored runs · 5 planned slots · 0 awaiting judgment
- build Actions
- 60.00 meanMedian 65.00 · n=5
Range 28.00–90.00 · SD 26.82 - acknowledged
- 5.00
- needs Review
- 0.00
Challenge telemetry
Claude Opus 5.5
Challenge telemetry
Claude Opus 5.5
Section results, sample counts, and the extra metrics recorded by each challenge.
Creativity
79.30 / 100 · 5 scored runs · 5 planned slots · 0 awaiting judgment
- build Actions
- 6.60 meanMedian 4.00 · n=5
Range 0.00–24.00 · SD 9.99 - acknowledged
- 5.00
- needs Review
- 0.00
Challenge telemetry
Gemini 4 Argon
Challenge telemetry
Gemini 4 Argon
Section results, sample counts, and the extra metrics recorded by each challenge.
Creativity
77.62 / 100 · 5 scored runs · 5 planned slots · 0 awaiting judgment
- build Actions
- 64.60 meanMedian 73.00 · n=5
Range 47.00–79.00 · SD 14.91 - acknowledged
- 5.00
- needs Review
- 0.00
Challenge telemetry
GPT-6.1 Terra
Challenge telemetry
GPT-6.1 Terra
Section results, sample counts, and the extra metrics recorded by each challenge.
Creativity
75.33 / 100 · 5 scored runs · 5 planned slots · 0 awaiting judgment
- build Actions
- 68.60 meanMedian 52.00 · n=5
Range 27.00–124.00 · SD 38.08 - acknowledged
- 5.00
- needs Review
- 1.00
Challenge telemetry
Claude Fable 5.1
Challenge telemetry
Claude Fable 5.1
Section results, sample counts, and the extra metrics recorded by each challenge.
Creativity
64.60 / 100 · 5 scored runs · 5 planned slots · 0 awaiting judgment
- build Actions
- 21.00 meanMedian 17.00 · n=5
Range 1.00–53.00 · SD 19.81 - acknowledged
- 5.00
- needs Review
- 0.00
Challenge telemetry
Gemini 3.8 Flash
Challenge telemetry
Gemini 3.8 Flash
Section results, sample counts, and the extra metrics recorded by each challenge.
Creativity
54.03 / 100 · 5 scored runs · 5 planned slots · 0 awaiting judgment
- build Actions
- 42.00 meanMedian 45.00 · n=5
Range 20.00–61.00 · SD 15.43 - acknowledged
- 5.00
- needs Review
- 1.00
Challenge telemetry
Claude Sonnet 5.5
Challenge telemetry
Claude Sonnet 5.5
Section results, sample counts, and the extra metrics recorded by each challenge.
Creativity
48.28 / 100 · 5 scored runs · 5 planned slots · 0 awaiting judgment
- build Actions
- 20.40 meanMedian 11.00 · n=5
Range 0.00–52.00 · SD 23.39 - acknowledged
- 5.00
- needs Review
- 0.00
Challenge telemetry
Muse Spark 1.3
Challenge telemetry
Muse Spark 1.3
Section results, sample counts, and the extra metrics recorded by each challenge.
Creativity
42.76 / 100 · 5 scored runs · 5 planned slots · 0 awaiting judgment
- build Actions
- 40.20 meanMedian 38.00 · n=5
Range 35.00–49.00 · SD 5.36 - acknowledged
- 5.00
- needs Review
- 0.00
Challenge telemetry
Muse Spark Open
Challenge telemetry
Muse Spark Open
Section results, sample counts, and the extra metrics recorded by each challenge.
Creativity
37.49 / 100 · 5 scored runs · 5 planned slots · 0 awaiting judgment
- build Actions
- 49.80 meanMedian 66.00 · n=5
Range 9.00–81.00 · SD 32.10 - acknowledged
- 5.00
- needs Review
- 1.00
Challenge telemetry
GPT-6 Luna
Challenge telemetry
GPT-6 Luna
Section results, sample counts, and the extra metrics recorded by each challenge.
Creativity
36.96 / 100 · 5 scored runs · 5 planned slots · 0 awaiting judgment
- build Actions
- 31.80 meanMedian 22.00 · n=5
Range 12.00–53.00 · SD 19.27 - acknowledged
- 5.00
- needs Review
- 0.00
Challenge telemetry
DeepSeek V4.1 Flash
Challenge telemetry
DeepSeek V4.1 Flash
Section results, sample counts, and the extra metrics recorded by each challenge.
Creativity
21.58 / 100 · 5 scored runs · 5 planned slots · 0 awaiting judgment
- build Actions
- 6.40 meanMedian 2.00 · n=5
Range 0.00–20.00 · SD 8.65 - acknowledged
- 4.00
- needs Review
- 0.00
Challenge telemetry
Nemotron 3.5 Super VL
Challenge telemetry
Nemotron 3.5 Super VL
Section results, sample counts, and the extra metrics recorded by each challenge.
Creativity
8.14 / 100 · 5 scored runs · 5 planned slots · 0 awaiting judgment
- build Actions
- 16.20 meanMedian 14.00 · n=5
Range 11.00–24.00 · SD 5.26 - acknowledged
- 5.00
- needs Review
- 0.00
Challenge telemetry
MiniMax M3.1 Flash
Challenge telemetry
MiniMax M3.1 Flash
Section results, sample counts, and the extra metrics recorded by each challenge.
Creativity
7.55 / 100 · 5 scored runs · 5 planned slots · 0 awaiting judgment
- build Actions
- 6.60 meanMedian 2.00 · n=5
Range 0.00–24.00 · SD 10.14 - acknowledged
- 3.00
- needs Review
- 1.00
Challenge telemetry
Gemma 4 31B
Challenge telemetry
Gemma 4 31B
Section results, sample counts, and the extra metrics recorded by each challenge.
Creativity
7.30 / 100 · 5 scored runs · 5 planned slots · 0 awaiting judgment
- build Actions
- 35.20 meanMedian 21.00 · n=5
Range 17.00–78.00 · SD 25.59 - acknowledged
- 5.00
- needs Review
- 2.00
Challenge telemetry
GLM 5.3
Challenge telemetry
GLM 5.3
Section results, sample counts, and the extra metrics recorded by each challenge.
Creativity
1.86 / 100 · 5 scored runs · 5 planned slots · 0 awaiting judgment
- build Actions
- 7.60 meanMedian 0.00 · n=5
Range 0.00–27.00 · SD 11.84 - acknowledged
- 2.00
- needs Review
- 0.00
Challenge telemetry
Grok 4.7
Challenge telemetry
Grok 4.7
Section results, sample counts, and the extra metrics recorded by each challenge.
Creativity
0.00 / 100 · 5 scored runs · 5 planned slots · 0 awaiting judgment
- build Actions
- 0.00 meanMedian 0.00 · n=5
Range 0.00–0.00 · SD 0.00 - acknowledged
- 0.00
- needs Review
- 0.00