Bottom line: worth watching and selectively benchmarking, but not adopting wholesale yet. The open-source skill is mostly a persistent brevity prompt; independent tests show it does not clearly beat “Be brief.” and yields about 8.5% output-token savings on realistic coding-agent work, far below the 65% prose-heavy headline. The strategically interesting layer is the still-private engine: context/tool-schema compression, spend attribution, caching, model routing, and eval-gated rollout. [Source: compiled from Caveman README, Max Taylor benchmark, JetBrains benchmark coverage, and Caveman product reports]
Source: https://caveman.so/ Repository: https://github.com/JuliusBrussee/caveman Author: Julius Brussee Ingested: 2026-07-26
What it is
Caveman currently has two materially different layers. The public MIT-licensed skill tells coding agents to remove filler and preserve code, commands, URLs, paths, and errors; the commercial Caveman Engine claims local, recoverable context compression and is explicitly “in private development.” [Source: Caveman homepage, https://caveman.so/; Caveman Engine, https://caveman.so/products/caveman-engine]
The broader product thesis spans output brevity, persistent context compression, MCP/tool-description compression, cost attribution, prompt caching, model routing, and eval-gated rollout. [Source: Caveman README, https://github.com/JuliusBrussee/caveman; Caveman homepage, https://caveman.so/]
Evidence
- Caveman’s own benchmark reports 65% average output-token reduction across ten prose-heavy prompts, with a 22–87% range. It explicitly does not reduce hidden reasoning, input context, files, or tool output. [Source: Caveman README, https://github.com/JuliusBrussee/caveman]
- The skill itself adds about 1–1.5k input tokens per turn. Caveman’s own “Honest Numbers” page says terse coding Q&A can therefore become net-negative and suggests ordinary replies need to exceed roughly 1.5–2k output tokens before the skill probably saves money. [Source: Caveman Honest Numbers, https://github.com/JuliusBrussee/caveman/blob/main/docs/HONEST-NUMBERS.md]
- Max Taylor’s independent 24-prompt, five-arm test found “Be brief.” at 419 mean tokens versus 401/404/449 for Caveman lite/full/ultra, with all arms within 1.5% quality. His conclusion: brevity alone is not the differentiator; consistent output structure and persistence are. [Source: Max Taylor, “I benchmarked caveman against two words,” https://www.maxtaylor.me/articles/i-benchmarked-caveman-against-two-words]
- A JetBrains-linked benchmark on 86 SkillsBench coding tasks reported about 8.5% output-token savings with statistically indistinguishable task quality; code, diffs, tool calls, and errors dominate agentic token streams and are intentionally left untouched. [Source: TechTimes summary of JetBrains benchmark, https://www.techtimes.com/articles/320756/20260716/jetbrains-tests-caveman-token-skill-86-real-tasks-savings-hit-9-not-65.htm]
- The repository was created on 2026-04-04 and had about 89.9k stars, 5.2k forks, 398 open issues, and an MIT license in the GitHub API snapshot inspected on 2026-07-26. [Source: GitHub API, https://api.github.com/repos/JuliusBrussee/caveman]
- Caveman’s CaveBench methodology is thoughtful—savings at held quality, cost per correct task, multi-seed confidence intervals, signed receipts—but its public head-to-head leaderboard is not live. [Source: Caveman Labs, “CaveBench methodology,” https://caveman.so/labs/articles/cavebench-methodology]
See also
- Julius Brussee
- Caveman GitHub repository
- CaveBench methodology