ai-research-reproduction Skill: Reproduce ML Repos From the README

ai-research-reproduction makes your agent run the smallest documented target of a deep learning repo and record auditable evidence of what happened.

Installs: 345K; on skills.sh
Stars: 486; on GitHub
Files: 47; besides SKILL.md
Updated: Sep 2026; last repo commit

The skill reads the repo README first, picks the smallest trustworthy target (documented inference, then evaluation, then training startup; full training only after you confirm), and runs it through a plan, run and verify flow. Evidence goes to repro_outputs/, including SUMMARY.md, COMMANDS.md, LOG.md, status.json and an annotated copy of the README.

It prefers no repo edits, records any patch it must make, and counts a result as matched only against expected metrics you give it. It needs Python 3.11+ and Git, and the target repo may need more dependencies, network access or a GPU.

Good for

ML researchers who want a paper repo run from its README with a clear record
Engineers checking whether a repo's documented inference or evaluation works
Teams that need assumptions, patches and deviations written down, not hidden

Install ai-research-reproduction

Claude Code

Copy this prompt.

Install the ai-research-reproduction skill in Claude Code. Run this command:

npx skills add https://github.com/lllllllama/rigorpilot-skills --skill ai-research-reproduction

Then check that it loaded and tell me how to use it.

2
Paste it into Claude Code. It shows the change and asks before running it.

Requires Node.js.

From its listing on skills.sh. Source.

What's in the skill

When it loads

Rigor Reproduce compatible skill slug for README-first deep learning repository reproduction. Use when the user wants an end-to-end, minimal-trustworthy flow that reads the repository first, selects the smallest documented inference or evaluation target, coordinates intake, setup, trusted execution, optional trusted training, optional repository analysis, and optional paper-gap resolution, enforces conservative patch rules, records evidence assumptions deviations and human decision points, and writes the standardized `repro_outputs/` bundle. Do not use for paper summary, generic environment setup, isolated repo scanning, standalone command execution, silent protocol changes, score chasing, or broad research assistance outside repository-grounded reproduction.

Sections in SKILL.md

Purpose
Fast Path
Fit
Trusted Target Selection
Workflow
Patch Boundary
Outputs
Reference Loading

Key facts

Author: lllllllama (owner of the lllllllama/rigorpilot-skills repo)
License: MIT
Installs on skills.sh: 345,030 as of 2026-11-02
GitHub stars: 486 for the whole lllllllama/rigorpilot-skills repo
skills.sh audits: Gen Agent Trust Hub pass, Socket warn, Snyk pass
What is inside: 47 files besides SKILL.md (18 in _bundled/, 16 in references/, 7 in assets/, 5 in scripts/, 1 in agents/)
Needs: Python 3.11+ and Git for the bundled orchestration; the target repo may need extra dependencies, network access or accelerators
Last commit to the repo: 2026-09-23

What it includes

Lists README commands and picks the smallest target
Runs scripts/orchestrate_repro.py in plan mode. It returns command candidates, the selected one and a fingerprint, and does not run the target.
Try: “Use ai-research-reproduction: run the smallest README-documented evaluation and write evidence to repro_outputs/.”
Runs code · Plan (--plan-only)
Runs one reviewed command tied to the plan fingerprint
Uses --run-selected with the command id and fingerprint. A changed command set fails before anything runs, and setup or download commands are never chosen as targets.
Runs code · Run the selected command
Rechecks the saved evidence without rerunning
Runs --verify-output. Detailed logs are read only when verification fails or the result is partial or blocked.
Read · Verify the output
Checks your environment on first use
Diagnoses install and environment problems before a first run.
Runs code · scripts/doctor.py
Inference, then evaluation, then training startup
Full training waits for your explicit confirmation. The agent also pauses before changing datasets, splits, checkpoints, metrics, loss or model meaning.
Read · Target ladder
Avoids edits; logs any it makes
Tries arguments, environment variables, path and dependency fixes before code changes. If files must change it uses a repro/YYYY-MM-DD-short-task branch and records them in PATCHES.md.
Write · Patch boundary
Adds per-section evidence next to the original README
Writes ANNOTATED_README.md, keeping the source README unchanged outside the inserted evidence blocks.
Write · Annotated README

From the official docs

Permissions

Medium risk

The skill runs commands from a repo on your machine with no sandbox. Use repos you trust and review the plan before the run step.

What it can do

Read your data
Create and change things
Delete things
Read-only mode

Watch out for

It runs code from the repo you point it at
The README says this is local execution, not an OS sandbox, and approved commands can reach your host and network. Use repos you trust and read the plan first. github.com
It can create branches and patch files
Edits are meant to be rare and are logged in PATCHES.md on a repro/ branch. Check the diff before keeping them.
Long runs use your GPU and time
Full training needs your confirmation. Set a command timeout if you want a hard limit.

Technical details

Talks to	Whatever the target repo's commands use
Scopes	Runs the target repo's documented commands and the skill's Python scripts Writes repro_outputs/ and can create a repro/ branch with patches May download data or use the network if the README commands do

Questions

What does ai-research-reproduction do?

It runs a deep learning repository from its README in a bounded way and writes evidence of the outcome. It starts from the smallest documented target and records assumptions, deviations and blockers instead of forcing success.

How do I install it?

Use the install box at the top of this page. The README says this main skill works alone, and that the "all skills" install adds the companion and leaf skills. The README says the installer needs Node.js/npm and that skills@1.5.26 requires Node 22.20.0 or newer.

Does it need Python?

Yes, Python 3.11+ and Git, per its compatibility note. The target repo may need more, such as extra packages, downloads or a GPU.

Does it need an API key?

No key is named in SKILL.md. The README describes an optional standalone model runner, which is separate from using the skill in your agent.

Does it prove a paper result?

No. A process that finishes is not counted as reproduction. SKILL.md awards a result match only against explicit expected metrics and a tolerance, and the README says green does not automatically mean paper-result reproduction.

How is it different from ai-research-explore?

This skill is for trusted, README-first reproduction. ai-research-explore is for candidate-only experiments that you explicitly authorize on top of a research anchor.

Is it safe?

It runs repo commands on your machine, and the README says this is local execution, not an OS sandbox. Use trusted repos and review the plan before the run step. See the audit results under key facts.

Information

updated Nov 2, 2026

Repository: lllllllama/rigorpilot-skills
Folder: skills/ai-research-reproduction

lllllllama/rigorpilot-skills skills.sh

Sources

Docs, issue trackers and discussions this page is built from.

Show all 3

ai-research-reproduction SKILL.md · github.com
RigorPilot Skills README · github.com
skills.sh listing and audits · skills.sh