Anthropic skill-creator: write, test and improve Claude skills

Anthropic's skill-creator helps you write a new Claude skill, test it against Claude without the skill, and improve it from the scores.

Installs: 421.3K; on skills.sh
Stars: 186.4K; on GitHub
Files: 17; besides SKILL.md
Updated: Oct 2026; last repo commit

skill-creator makes Claude interview you about a task, write the SKILL.md, then test it against a run without the skill and show you the scores. You review the outputs in a browser page, give feedback, and Claude rewrites the skill. You can also stop after the first draft and skip the tests.

It works fully in Claude Code, where Claude can start separate helper runs for each test. In Claude.ai it still interviews you, writes the skill and packages it, but tests run one at a time. The baseline comparison, benchmark numbers, blind comparison and description tuning are skipped there.

You need Python 3 for the bundled scripts, and PyYAML for validating and packaging. Description tuning also needs the claude command line tool. It is Anthropic's own skill, and it can use a lot of your plan's usage when it tunes descriptions (see the safety answer in the FAQ).

This page covers the skill itself. For the full guide to writing a skill by hand, see how to create Claude skills. The same skill ships as the skill-creator plugin in Anthropic's official Claude Code marketplace.

Good for

Anyone turning a repeated Claude task into a reusable skill
Claude Code users who want proof a skill helps before they share it
Teams keeping a library of skills and checking edits do not break them
Skill authors whose skill triggers too rarely or at the wrong time

Install skill-creator

Claude Code

Copy this prompt.

Install the skill-creator skill in Claude Code. Run this command:

/plugin install skill-creator@claude-plugins-official

Then check that it loaded and tell me how to use it.

2
Paste it into Claude Code. It shows the change and asks before running it.

Requires Python 3.

Full evals need subagents and the claude CLI, which Claude Code has. Packaging needs PyYAML.

From the vendor's docs. Source.

What's in the skill

When it loads

Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.

Sections in SKILL.md

Communicating with the user
Creating a skill
Capture Intent
Interview and Research
Write the SKILL.md
Skill Writing Guide
Report structure
Executive summary
Key findings
Recommendations
Commit message format
Writing Style
Test Cases
Running and evaluating test cases
Step 1: Spawn all runs (with-skill AND baseline) in the same turn
Step 2: While runs are in progress, draft assertions

Key facts

Author: Anthropic
License: Apache-2.0 (LICENSE.txt in the skill folder)
Evals update: 2026-03-03 (evals, benchmarks, description tuning)
Skill folder last changed: 2026-04-20
Installs on skills.sh: 421,179 as of 2026-11-02
Requires: Python 3 and PyYAML; helper runs and the claude CLI for full evals
Works in: Claude Code (full), Claude.ai (partial), Cowork
skills.sh audits: Gen Agent Trust Hub pass, Snyk pass, Socket warn (scanned 2026-09-15)
Also packaged as: skill-creator plugin in claude-plugins-official

What it includes

Asks what the skill should do and when it should run
Claude asks what the skill does, when it should trigger, the output format and whether to set up tests. If you say 'turn this into a skill' mid-chat, it answers these from the chat first.
Try: “turn this into a skill”
Read · 1. Capture intent
Asks about edge cases, formats and success criteria
Claude asks about edge cases, input and output formats, example files and dependencies. It can use connected tools or helper runs to research.
Read · 2. Interview and research
Drafts the skill file, name and description
Claude writes the name, a deliberately pushy description to avoid under-triggering, and the body. The guide says to keep it under 500 lines and move detail into separate files.
Write · 3. Write the SKILL.md
Saves 2 or 3 realistic test prompts
You confirm a few realistic prompts and Claude saves them to evals/evals.json.
Write · 4. Test cases
Runs each test with and without the skill
Each test runs in two helper runs at once, one with the skill and one without (or with your old version). Tokens and time are recorded. Not available in Claude.ai.
Runs code · 5. With-skill and baseline runs
Scores results and opens a review page
A grader checks each assertion, a script rolls up pass rate, time and tokens with average, spread and difference, and a viewer shows outputs and benchmark tabs. You leave feedback there.
Runs code · 6. Grade, benchmark and review
Rewrites the skill from your feedback
Claude generalizes your feedback, keeps the prompt lean and reruns into a new iteration folder. It stops when you are happy, feedback is empty or progress stalls.
Write · 7. Improve and rerun
Judges two versions without labels
An optional judge compares outputs from two versions without knowing which is which, then another agent explains why the winner won. Needs helper runs.
Try: “is the new version actually better?”
Read · 8. Blind comparison
Tests and rewrites the description to trigger better
Runs 20 sample requests 3 times each, splits 60/40 train and test, and rewrites the description for up to 5 rounds. Needs the claude CLI and can use a lot of plan usage.
Try: “Optimize the description of my invoice-parser skill.”
Runs code · 9. Description tuning
Zips the skill into a .skill file
Checks the skill, then zips it into a .skill file you can save or share. The evals folder is left out.
Write · 10. Package
Instructions for the grading helper
Checks each assertion against the transcript and outputs, writes pass or fail with evidence, and critiques weak assertions.
agents/grader.md
Instructions for the blind judge
Compares two outputs without knowing which skill made them.
agents/comparator.md
Instructions for the analysis helper
Explains why a comparison winner won and flags assertions that do not tell versions apart or that flip between runs.
agents/analyzer.md
Builds and serves the review page
Builds the review page from a workspace and serves it on 127.0.0.1 (port 3117 by default), or writes a static HTML file with --static. Saves your feedback to feedback.json.
Runs code · eval-viewer/generate_review.py
Rolls grading results into benchmark numbers
Produces benchmark.json and benchmark.md with mean, spread, min, max and the with-versus-without difference.
Write · scripts/aggregate_benchmark.py
Runs the description tuning loop
Splits queries into train and test sets, runs the trigger test, asks Claude for better descriptions and returns the best one by test score. Needs the claude CLI.
Runs code · scripts/run_loop.py
Tests whether a request triggers the skill
Writes a temporary command file into the project's .claude/commands folder, runs claude -p with each query, checks whether the skill was used and deletes the file.
Runs code · scripts/run_eval.py
Asks Claude for a better description
Calls claude -p with the failures so far and returns a new description to try.
Runs code · scripts/improve_description.py
Makes the HTML report for tuning
Shows each description attempt with train and test scores.
Write · scripts/generate_report.py
Validates and zips a skill
Creates the .skill file after validation.
Write · scripts/package_skill.py
Page to review trigger test queries
An HTML page where you edit the trigger query set and export eval_set.json.
assets/eval_review.html
Used by the assistant behind the scenes
Checks SKILL.md frontmatter
Checks that SKILL.md exists and has valid frontmatter. Needs PyYAML.
Read · scripts/quick_validate.py
Reads SKILL.md frontmatter
Shared helper that parses frontmatter.
scripts/utils.py
File format reference
Describes the JSON files used for evals, grading, benchmark and timing.
references/schemas.md

From the official docs

Permissions

Medium risk

What it can do

Read your data
Create and change things
Delete things
Read-only modeHow to turn it on

Watch out for

Runs many claude -p sessions on your plan
Description tuning runs each of about 20 queries 3 times per round for up to 5 rounds via the claude CLI; one user reports it used a whole Max plan's quota in minutes. github.com
Starts a local web server for the review page
The eval viewer serves results on 127.0.0.1 (default port 3117) and saves your feedback to feedback.json; a static HTML mode exists for headless setups. github.com
Viewer stops whatever is on its port
Before starting, the viewer sends SIGTERM to any process listening on the chosen port without asking, so a custom --port can kill an unrelated app. github.com
Eval output can inject script into the viewer
Reported and still open: test outputs containing </script> or crafted HTML can run script in the review page and overwrite feedback. Socket flags the same viewer file. github.com
Writes temporary command files in .claude/commands
Trigger tests write a temporary command file per query into the project's .claude/commands folder and delete it afterwards. github.com
Viewer loads SheetJS and Google Fonts from CDNs
viewer.html pulls SheetJS from cdn.sheetjs.com and fonts from Google, so opening the review page makes outbound requests. skills.sh
skills.sh audits: 2 pass, 1 warn
Gen Agent Trust Hub: pass (safe). Snyk: pass, no issues. Socket: warn, one low-severity anomaly in eval-viewer/viewer.html (possible DOM XSS, unsandboxed data: iframe). Scanned Sep 15, 2026. skills.sh

Security advisories

No published security advisories as of Nov 2, 2026.

Technical details

Turning on read-only	Skip the eval and description-tuning steps (the skill allows a 'just vibe with me' mode) and Claude only drafts files; in Claude Code, normal permission prompts still gate each script and subagent.
Talks to	127.0.0.1 (local review page). cdn.sheetjs.com and Google Fonts (review page)
Scopes	Runs Python scripts Spawns subagents and claude -p sessions Starts a local web server Writes files in the skill folder and .claude/commands

What people say

People like that it tests a skill against no skill, and dislike its verbose drafts and unreliable description tuning.

What people like

Runs paired helper runs with and without the skill and judges whether they differ.
Works as a decent starting point for a first draft or a more tailored eval setup.
One blind test showed a real lift (84% to 100% pass rate) and found a section with no benefit to trim.
Commentators welcomed treating skills like software that needs testing.

Common complaints

Its instructions are seen as micromanaging; one user did better starting minimal.
In one three-round run the description optimizer never beat the starting description.
Trigger tests can report 0% recall because of test problems and waste plan usage.
The built-in report leaves out side-by-side outputs, per-run tokens and conversation logs.

In their words

Hacker NewsMay 2026

“Depending on skill, Claude already does this when creating new skills with their skill-creator skill (what a sentence), it's pretty neat. It creates ~6 subagents with and without the skill and judges if they differ in performance.”

ChairmanLmao Hacker NewsMay 2026

“The claude provided skill-creator provides a decent jumping off point. It is easy enough to start with, but unless the skill is really simple I found it best to consider it a scaffold for building more tailored evals and reports. The report leaves out a lot of detail.”

dsmmcken Hacker NewsSep 2026

“I had more success starting from scratch. Especially claude's skill-creator skill, it micromanages, which is in fact worse for 5+ models than just leaving the instructions out and crossing your fingers.”

plaguuuuuu Hacker NewsJun 2026

“I don’t write my own - I can’t optimize for the models understanding, and so I just give the skill-creator skill an outline and then have it refine until the output is what I want.”

threecheese

3 more quotes

BlogMay 2026

“The description optimizer ran for three iterations and never beat the description I started with. And my evals are now saturated at 100%, which is itself a problem.”

Daniel Sogl BlogMar 2026

“This isn't just an incremental capability - it's a big step in treating skills and context as a piece of software, and one that requires testing to get right.”

Baptiste Fernandez GitHub issueJul 2026

“The description-optimization loop in skill-creator (scripts/run_loop.py → scripts/run_eval.py) consumed 100% of a Claude Max subscription's quota in roughly 3 minutes (121 claude -p invocations before crashing in iteration 2).”

guala19

Examples

Test an existing skill in Claude Code (from Claude Code docs)
evaluate my summarize-changes skill with skill-creator
Claude runs your test prompts with and without the skill and shows the results.
Source
Start a new skill in Claude.ai (from Anthropic's tutorial)
I want to create a skill for quarterly business reviews
Claude interviews you and drafts the skill.
Source
Turn a workflow you just did into a skill (phrase from SKILL.md)
turn this into a skill
Claude pulls the steps and corrections from the chat first, then asks what is missing.
Source
Check an edit helped, with blind comparison (phrase from SKILL.md)
is the new version actually better?
A judge compares the two outputs without knowing which skill made which. Needs helper runs.
Source
Tune when a skill triggers (Claude Code only)
Use skill-creator to optimize the description of my invoice-parser skill so it triggers on real invoice requests and not on general PDF questions.

Questions

How do I use skill-creator in Claude Code?

Install it with the command in the install box, then ask Claude to use it. Anthropic's docs give the example evaluate my summarize-changes skill with skill-creator, or say create a skill that ... to start a new one. Claude interviews you, writes the skill, runs test prompts and opens a review page. If the install says the marketplace is not found, add anthropics/claude-plugins-official with /plugin marketplace add and try again. If it says the plugin is not found, run /plugin marketplace update claude-plugins-official.

How do I use skill-creator in Claude.ai?

Anthropic says the update is available in Claude.ai and Cowork. Turn on code execution and file creation in your settings, then ask Claude to use skill-creator, for example I want to create a skill for quarterly business reviews. Claude interviews you, writes the skill and packages a .skill file you can save and turn on. Anthropic's help center does not describe a separate skill-creator switch, so if it is not available, check your skills list under Customize > Skills. The repo README says its example skills are already available to paid plans.

In Claude.ai there are no helper runs, so see the next answer for what is skipped.

Does skill-creator work the same in Claude.ai as in Claude Code?

No. Claude.ai cannot start separate helper runs, so tests run inline one at a time. Baseline runs, benchmark numbers, blind comparison and description tuning are skipped. Description tuning also needs the claude command line tool, which comes with Claude Code. Packaging a finished skill still works in Claude.ai. Cowork runs the full loop but has no display, so the review page is saved as a file and your feedback downloads as feedback.json.

How do I install skill-creator?

Use the install box above. In Claude Code the documented route is /plugin install skill-creator@claude-plugins-official. The other route is Anthropic's skills repo: add it with /plugin marketplace add anthropics/skills, then run /plugin install example-skills@anthropic-agent-skills. That plugin also installs the other example skills. The skills CLI command in the box copies just this skill. In Claude.ai there is nothing to install if it is already in your skills list.

How do skill-creator's evals and benchmarks work?

You agree on 2 or 3 realistic test prompts, saved in evals/evals.json. For each one Claude runs two helper runs at the same time, one with the skill and one without it (or with your old version). It then checks each result against assertions, rolls the scores into a benchmark, and opens a review page. The benchmark shows pass rate, time and tokens for both, with the average, spread and the difference. You read the outputs, leave feedback and Claude updates the skill for the next round. Results are saved in a <skill-name>-workspace folder next to the skill, one folder per round.

What is description tuning?

It tests whether Claude picks the skill at the right moments. Claude writes 20 sample requests, about half that should trigger the skill and half that are near misses. You review them in an HTML page. Then a script runs each request 3 times, keeps 60% for training and 40% for a final check, and asks Claude to rewrite the description for up to 5 rounds. It returns the description that scored best on the held-out 40%. Anthropic says it improved triggering on 5 of 6 of its public document skills. It needs the claude command line tool and no longer needs an ANTHROPIC_API_KEY.

Why does description tuning say my skill never triggers?

Several open GitHub reports say the trigger test can score 0% for skills that do trigger. Reported causes include parallel test runs interfering with each other and Windows problems. One user says the loop used a whole Max plan's usage in about 3 minutes. Treat 100% precision with 0% recall as a likely test problem and check by hand before accepting a new description. See issue 1721, issue 1706, issue 1692 and issue 1412. These are user reports, not confirmed by Anthropic.

Is skill-creator safe?

It is Anthropic's own skill under Apache 2.0, and two of three skills.sh audits pass: Gen Agent Trust Hub (safe) and Snyk (no issues). Socket warns with one low-severity alert about the review page file eval-viewer/viewer.html, a possible script injection. Socket's own wording is that it found no clear signs of malware. Running it means local Python scripts, helper runs, nested claude -p sessions, and a review server on 127.0.0.1 that saves your feedback. Open reports cover script injection in that page (issue 1982) and the server stopping whatever else uses its port (issue 6193). Trigger tests also write and delete temporary files in your project's .claude/commands folder. Review prompts and outputs you did not write, and use --static for a plain HTML page with no server. No security advisories are filed for the repo.

Should I use skill-creator or write SKILL.md by hand?

Use skill-creator if you want measured results, or a first draft from an interview. Write by hand if you want a short, minimal skill. One Hacker News user says the skill-creator instructions micromanage and they did better starting from scratch. Another uses it to refine from an outline. A good mix is to write a small skill yourself and use its tests to check changes. For the full guide to writing a skill by hand, see how to create Claude skills.

Does skill-creator work in Codex, Cursor, OpenCode or OpenClaw?

Each has its own version. Codex has a built-in $skill-creator. Cursor has /create-skill and also reads .claude/skills. OpenCode has a community port, antongulin/opencode-skill-creator, which calls itself an independent adaptation of Anthropic's. OpenClaw bundles its own skill-creator, which validates and packages skills but has no test loop. Anthropic's scripts call the claude command line tool, and nobody documents running its test loop in Codex or Cursor.

Is skill-creator the same as the skill-creator plugin?

Yes, the content is the same. The skill-creator plugin in Anthropic's official marketplace carries an identical SKILL.md (checked 2026-11-02). Claude Code also has its own claude plugin eval command for plugin skills, which uses a different test format from skill-creator's evals/evals.json.

Information

updated Nov 2, 2026

Repository: anthropics/skills
Folder: skills/skill-creator

anthropics/skills skills.sh

Sources

Docs, issue trackers and discussions this page is built from.

Show all 33

Using Skills in Claude · support.claude.com
Improving skill-creator: Test, measure, and refine Agent Skills · claude.com
Agent skills · code.claude.com
anthropics/skills README · github.com
skills.sh · skills.sh
skill-creator SKILL.md · github.com
GitHub · github.com
skill-creator LICENSE.txt · github.com
Commits · github.com
GitHub · github.com
skills.sh · skills.sh
skills.sh · skills.sh
skills.sh · skills.sh
GitHub · github.com
0% trigger rate · github.com
Parallel trigger tests · github.com
Windows · github.com
GitHub · github.com
github.com · github.com
github.com · github.com
Hacker News · news.ycombinator.com
Hacker News · news.ycombinator.com
OpenAI · learn.chatgpt.com
Cursor · cursor.com
GitHub · github.com
GitHub · github.com
github.com · github.com
github.com · github.com
Hacker News · news.ycombinator.com
Hacker News · news.ycombinator.com
Blog · dev.to
Blog · tessl.io
Claude Academy · academy.claude.com