caveman-optimize Skill for Claude: Test One Cost-Saving Change With a Paired Eval

caveman-optimize makes your coding assistant test one token-saving change against a baseline, but only after you pick and approve it.

Installs: 112.6K; on skills.sh
Stars: 116.7K; on GitHub
Updated: Nov 2026; last repo commit

caveman-optimize reads the report-only observations from the caveman command line tool, such as how big your context window or tool output is. It shows them unranked and waits for you to choose one. Then it looks for a real mechanism in your code, proposes one small change and a paired test, and applies it only after you approve.

It needs a logged-in Caveman CLI session, and it stops without editing if the CLI, the login or the observations are missing. It reports observations, not savings: the SKILL.md says the profiles carry a zero opportunity band and that a local result does not prove production savings. It bundles no scripts.

Good for

Developers who use Caveman and want to act on its usage reports
Teams that want a baseline-versus-candidate test before keeping a cost change
Anyone who wants no savings claims without evidence

Install caveman-optimize

Claude Code

Copy this prompt.

Install the caveman-optimize skill in Claude Code. Run this command:

npx @caveman-ai/cli@latest

Then check that it loaded and tell me how to use it.

2
Paste it into Claude Code. It shows the change and asks before running it.

Requires Node.js 22.13+.

This skill drives the caveman command line tool, so the skill file alone is not enough. The README lists it under the commands you use in your agent after the full install. Run caveman login before first use. Pick one route, not both.

From the project README. Source.

What's in the skill

When it loads

Turn a Caveman optimization observation into an operator-chosen candidate with a paired baseline evaluation. Use when asked to inspect or evaluate a Caveman optimization report. Needs explicit approval.

Sections in SKILL.md

1. Read the exact observations
2. Ask the operator to choose
3. Design a candidate and paired eval
4. Apply only the approved candidate
5. Report observations, not savings

Key facts

Author: juliusbrussee (GitHub)
Repository: juliusbrussee/caveman, folder skills/caveman-optimize
Installs on skills.sh: 112,627 (as of 2026-11-02)
Repo stars: 116,702 (whole repository, as of 2026-11-02)
License: Apache-2.0
Last repo commit: 2026-11-01
Files in the skill folder: 1 file: SKILL.md
skills.sh audits: Gen Agent Trust Hub: pass, Socket: pass, Snyk: pass

What it includes

Read the report-only observations from the Caveman CLI
Runs the CLI and reads only the report_only_observations list. Stops if the CLI, login or list is missing.
Runs code · caveman opportunities list
Show the observations unranked and wait for your pick
Shows each id, exact title, exact observation and last_seen_at, and asks for an explicit choice before looking at code.
Ask you to choose
One minimal change plus a baseline-vs-candidate test
Inspects your repo for a mechanism behind the observation and proposes one change with a fixed fixture, a quality check and the same cost measure for both runs.
Read · Design a candidate and paired eval
Edit only the evidenced callsite, then run both arms
Applies the approved change, runs the paired eval and the repo checks, and reverts it if quality drops or the result is unclear.
Runs code · Apply the approved candidate
Fixed report format; no dollar figures
Reports the observation, candidate, eval result, checks and a keep, reject or inconclusive decision, with no inferred savings.
Report observations, not savings

From the official docs

Permissions

Medium risk

It uses your Caveman login and can edit code in your repo after you approve a change. Use a branch, and approve the candidate and the eval design yourself.

What it can do

Read your data
Create and change things
Delete things
Read-only mode

Watch out for

The install route changes your agent setup
The README says the full Caveman install finds your agents and logins, shows what it will change and waits for you to press Enter. The Claude Code plugin route auto-starts caveman in every interactive session. The README's Privacy section says the CLI sends usage stats by default (commands run, token counts, a random install ID, OS and IP, not prompts, code or file paths); `caveman telemetry off` or DO_NOT_TRACK=1 turns it off.
It needs a Caveman login and edits your code
It runs the caveman CLI while you are logged in and may change a callsite in your repo after you approve. The SKILL.md does not say what the CLI sends or where. Work on a branch and review the diff.
It will not apply a change without your approval
The SKILL.md asks for an explicit choice and for approval of the candidate and the eval design before any edit. Do not skip those prompts.

Questions

What does caveman-optimize do?

It takes one report-only observation from the Caveman CLI, such as a context-window or tool-output profile, and tests one code change against a baseline on identical inputs. It reports whether the change is worth keeping. It does not claim dollar savings.

How do I install caveman-optimize?

Use the install box at the top of the page. This skill depends on the caveman command line tool and a login, so the box lists the full Caveman setup first. Then run caveman login.

Does caveman-optimize need an account or API key?

It needs a logged-in Caveman CLI session. The SKILL.md says not to fall back to a raw gateway Cave Plan or a project API key, because those do not provide the report-only observations. The README says the account is free.

Will it change my code on its own?

No. It asks you to choose an observation, then asks you to approve the candidate and the eval design before it edits. It will not create a Caveman experiment, mark anything implemented or switch on an optimizer.

Is caveman-optimize safe?

It can edit files in your repo and runs the Caveman CLI with your login, so use it on a branch and review the diff. The SKILL.md limits edits to the evidenced callsite and says to revert if quality drops or the comparison is unfair.

How is it different from caveman-discover and caveman-evidence-review?

caveman-optimize tests a cost change from a report-only observation. caveman-discover labels your LLM workflows so spend is grouped by job, and caveman-optimize hands off to it when the only item is unlabeled traffic. caveman-evidence-review reads Caveman Cloud evidence without editing code.

Information

updated Nov 2, 2026

Repository: juliusbrussee/caveman
Folder: skills/caveman-optimize

juliusbrussee/caveman skills.sh

Sources

Docs, issue trackers and discussions this page is built from.

Show all 4

caveman README · github.com
caveman-optimize SKILL.md · github.com
caveman-discover SKILL.md · github.com
caveman-evidence-review SKILL.md · github.com