radar

Gauntlet Loops: external quality bars and fresh-context critics

concept trial 8 August 2026

Gauntlet Loops: external quality bars and fresh-context critics

Bottom line: worth trialing inside moyn's devbox delivery loop. The useful innovation is not unlimited iteration or agent fan-out by itself; it is forcing builders to confront a concrete, inspectable external quality bar through a fresh-context critic that judges the real artifact. The public evidence also shows why the pattern needs budgets, measurable gates, coherent ownership, and human stopping rules.

Primary sources: Matt Shumer's definition on X, his detailed guide, the original Claude of Duty prompt, and the public Claude of Duty repository, accessed 2026-08-08.

What the pattern is

Matt Shumer describes a Gauntlet Loop as a lead agent that:

  1. receives an ambitious outcome and a real, inspectable example or metric for what excellent looks like;
  2. chooses its own approach and decomposes the artifact into the smallest parts that can be improved and judged independently;
  3. gives each important part to a specialist builder;
  4. sends the actual output—not the builder's explanation—to a separate critic with fresh context;
  5. asks the critic for a blind A/B choice against the external bar where possible;
  6. if the generated artifact loses, identifies the largest meaningful gap and routes it back for another build–inspect–judge cycle;
  7. stops when the output passes, the human stops it, improvement no longer justifies the cost, or an explicit time/cost/risk boundary fires.

Shumer's short formulation is: the agent, rather than the human, breaks the goal into parts; specialist builders face ruthless blind critics; and a part passes only if it beats a real-world equivalent. [Source: https://x.com/mattshumer_/status/2081830214384886228]

This is a specialized evaluator–optimizer loop, not a wholly new agent architecture. Its differentiators are the external bar, fresh critic, inspection of the real artifact, binary preference over drifting self-scores, and continuation without an arbitrary fixed round count.

Prompting principles around the loop

Shumer's broader prompting method predates the name and is documented in How I Prompt Fable:

A useful distinction: a normal coding loop is edit → run tests → read failure → revise. A Gauntlet adds an ambitious external comparator and an independent adversarial judge. For production use, it should still be wrapped in ordinary loop-engineering controls: objective, metrics, allowed actions, cost/time limits, retry strategy, durable state, escalation, and human approval for consequential changes.

Evidence and examples

1. Claude of Duty — origin and strongest inspectable case

A single short orchestration prompt asked Claude Code/Opus 5 to build a Three.js first-person shooter, fan out subagents, loop each component, and use harsh visual critics to compare it blind with current Call of Duty games. The resulting public project contains roughly 55,000 lines across 11 subsystems, with procedural textures, meshes, animation, and sound rather than imported art assets. It includes reproducible screenshot capture, per-pixel image-diff gates, scripted playtests, and performance profiling. [Source: repository README]

The repository's honest assessment matters more than the viral framing: every critic in every blind round still chose the real Call of Duty frame. Scores improved from 3.59 to 5.05/10, but most shots remained “AMATEUR.” The external bar acted as a direction and anti-complacency mechanism, not proof that the target was reached. [Source: repository README]

2. Der Koloss — applying the pattern to an existing game

Rishi (@0xRishi) ran an existing Three.js zombie game through an Opus 5 prompt derived from Shumer's approach. The thread shows the original version, the transformation prompt, the generated visual/gameplay overhaul, a playable multiplayer build, and later source publication. The author also reports rough edges after the first pass—useful evidence that one impressive transformation is not the same as passing a robust delivery gate.

3. Horror novel — non-code creative work

Shumer demonstrated a Gauntlet Loop writing a full horror novel while publishing progress to a live workbench. This shows the intended transfer beyond software: chapters or passages can be decomposed, built, and reviewed by separate fresh-context editors against concrete structural and clarity references. It is evidence of process use, not independently verified evidence that the final novel exceeded its benchmark.

4. Component-library visual cloning — agent invents the metric

Shumer describes abandoning a ShadCN-based attempt, rebuilding from scratch, and asking the agent to invent a measurable definition of visual parity. The agent recorded the reference components, converted movement into a heat map, and iterated until its version matched. The reusable point is to delegate metric discovery when the operator knows the desired outcome but not the correct verifier—then have a separate agent apply that verifier.

5. Community browser games

Shumer's guide collects playable adaptations of the prompt:

These demonstrate reproducibility of the broad prompting pattern, but quality claims vary and should not be treated as controlled benchmarks.

6. Reusable skill packaging

This third-party Claude skill turns a goal into a short prompt after proposing two or three named, fetchable, comparable quality bars. It usefully codifies three rejection criteria for a bad bar: it must be named, fetchable by the critic, and directly comparable. It also insists on fresh-context critics, binary preference instead of 1–10 scores, and exiting only when the artifact wins or the operator stops the run.

Deep dive: what Claude of Duty actually implemented

The initiating prompt is only 941 bytes. Its load-bearing elements are: an ambitious outcome, a named external reference, agent-chosen decomposition, separate harsh visual critics, blind side-by-side comparison, no fixed iteration count, and an agentic harness with /loop, subagents, and ultracode. It is not an architecture or implementation specification. [Source: https://github.com/mshumer/Claude-of-Duty/blob/main/prompt.md]

The public repository is an inspectable artifact and engineering postmortem, but not a complete replay of the original run. It does not contain the original Claude Code transcripts, critic prompts and responses, reference Call of Duty frames, blind-order records, or raw score sheets for all visual rounds. “One prompt” therefore means one operator instruction that launched a long autonomous tool-and-subagent run—not one model completion.

Architecture as a coordination contract

ARCHITECTURE.md was the central multi-agent contract. Eleven directory-owning domains—render, materials, sky, world, physics, player, weapons, FX, AI, UI, and audio—could work concurrently under rules prohibiting cross-domain edits and direct imports. Runtime dependencies were resolved through a registry and shared context/events. This reduced merge conflicts, but it did not make perceptually coupled systems independent: render, sky, exposure, tonemapping, lighting, and materials still changed one another's output. [Source: https://github.com/mshumer/Claude-of-Duty/blob/main/ARCHITECTURE.md]

The verification harness is the reusable artifact

The repository includes eleven canonical review shots plus two distinct capture modes:

This matters because earlier instruments gave confident but wrong answers. A static benchmark reported 94 FPS while actual motion ran at 12–17 FPS with 728–1236 ms stalls; fast captures differed on ten of eleven shots across identical runs. The lesson is to validate the measuring instrument against ground truth before letting it steer an autonomous loop. [Source: https://github.com/mshumer/Claude-of-Duty#tooling]

Results and topology lesson

Visual scores improved 3.59 → 4.14 → 4.05 → 5.05/10, but every blind critic still preferred the real Call of Duty frame. Three rounds of six parallel directory owners improved the score only +0.46 and ended with more severe defects. One sequential owner per coupled concern improved it +1.00 and reduced defects from 66 to 26. The useful pattern is therefore not “maximum fan-out”; it is parallel ownership for genuinely independent work, followed by coherent sequential ownership and integration for coupled concerns. [Source: https://github.com/mshumer/Claude-of-Duty#honest-assessment]

The best diagnostic also contradicted the critics' wording. Critics called the weapon “untextured”; direct measurement showed a specular-dominated lighting rig suppressing the diffuse contribution. Earlier literal fixes had lowered albedo and made the perceived problem worse. Critics should therefore identify observable gaps and reproduction evidence, while root-cause diagnosis remains a separate measured step.

Comparative prompt analysis and demos

Kart Royale — strongest documented replication

Der Koloss — impressive but not a clean one-prompt case

The Long Silence — genre transfer with strong inspection tooling

Pastel Nuketown — strong product evolution, weak provenance isolation

The Body We Agreed On — transparent non-code transfer

Prompt families: what survives across cases

  1. Canonical sparse prompt: names destination and external bar, delegates decomposition, separates builder and critic, and refuses arbitrary completion. Powerful for autonomy; dangerously underspecified on budgets, permissions, integration, and stop conditions.
  2. RoboNuggets skill: requires the bar to be named, fetchable, and directly comparable, then emits a short execution prompt. Useful for bar discovery; still optimistic about looping until a win. [Source: https://github.com/robonuggets/gauntlet-loop]
  3. Benchmark Loop: freezes shared foundations before fan-out, scopes a reachable comparison slice, randomizes blind ordering, asks critics for evidence and a “flip condition,” adds integration review, and defines termination conditions. This is a more operationally mature interpretation. [Source: https://github.com/martbln/benchmark-loop]
  4. Production playbook: retains sparse guidance about the solution path but makes scope, architecture contracts, evidence packs, budgets, security boundaries, human gates, observability, diminishing-return rules, and release criteria explicit. See Gauntlet Loops sicher und wirksam betreiben.

What the public case disproves or complicates

The Claude of Duty repository documents several failure modes that should be part of any adoption decision:

Related