PlugWiki

airflow-dag-patterns Skill for Claude: What It Does and How to Install It

Helps your assistant write Apache Airflow DAGs with sensible operators, sensors, tests and retries.

Installs
11.2K
on skills.sh
Stars
42.1K
on GitHub
Files
1
besides SKILL.md
Updated
Oct 2026
last repo commit

SKILL.md gives four DAG design principles (idempotent, atomic, incremental, observable), task dependency patterns (linear, fan-out, fan-in) and a quick-start daily ETL DAG with retries and exponential backoff. Its do and don't lists cover the TaskFlow API, sensor reschedule mode, depends_on_past and hardcoded dates.

Its "When to Use" list includes creating pipelines, custom operators and sensors, testing DAGs locally, setting up Airflow in production and debugging failed runs. Six worked patterns and a project layout sit in references/details.md. It bundles no scripts.

Good for

  • Data engineers building pipelines
  • Developers writing custom operators and sensors
  • Teams debugging failed DAG runs

Install airflow-dag-patterns

Claude Code

  1. 1

    Copy this prompt.

    Install the airflow-dag-patterns skill in Claude Code. Run this command:

/plugin marketplace add wshobson/agents
/plugin install data-engineering@claude-code-workflows

Then check that it loaded and tell me how to use it.
  2. 2

    Paste it into Claude Code. It shows the change and asks before running it.

Pick one route. The plugin routes install the whole data-engineering plugin: 2 agents, 2 commands and 4 skills (counted from the plugin folder in the repo tree, which has no hooks or MCP config files). The skills-only routes install just this skill, with no agents or commands. Running generated DAGs needs Apache Airflow in your own project.

From the project README. Source.

What's in the skill

When it loads

Build production Apache Airflow DAGs with best practices for operators, sensors, testing, and deployment. Use when creating data pipelines, orchestrating workflows, or scheduling batch jobs.

Sections in SKILL.md

  1. When to Use This Skill
  2. Core Concepts
  3. 1. DAG Design Principles
  4. 2. Task Dependencies
  5. Quick Start
  6. Detailed patterns and worked examples
  7. Best Practices
  8. Do's
  9. Don'ts

Key facts

Author
wshobson (repo owner)
License
MIT
Installs on skills.sh
11,171 (Nov 2, 2026)
GitHub stars
42,081 for the whole wshobson/agents repo (Nov 2, 2026)
Last repo commit
Oct 30, 2026
What it includes
2 files including SKILL.md: references/details.md (no scripts)
Needs
Apache Airflow in your project; the TaskFlow pattern is labelled Airflow 2.0+
Part of
The data-engineering plugin (version 1.3.3) in wshobson/agents. Counted from the plugin folder: 2 agents, 2 commands and 4 skills
Works in
Claude Code, Codex CLI, Cursor, OpenCode, Antigravity CLI, GitHub Copilot and Pi, per the README. Skills-only installers ask which tool you use

What it includes

  • Idempotent, atomic, incremental, observable

    Idempotent, atomic, incremental, observable.

    DAG design principles

  • Linear, fan-out, fan-in and mixed patterns

    Linear, fan-out, fan-in and mixed patterns.

    Task dependencies

  • A daily ETL DAG with retries, exponential backoff and an email-on-failure setting

    A daily ETL DAG with retries, exponential backoff and an email-on-failure setting.

    Quick start DAG

  • Use TaskFlow, set timeouts, reschedule-mode sensors, no depends_on_past

    Use TaskFlow, set timeouts, reschedule-mode sensors, no depends_on_past, no hardcoded dates.

    Best practices

  • Airflow 2.0+ decorators with automatic XCom

    Airflow 2.0+ decorators with automatic XCom (in references/details.md)

    Pattern 1: TaskFlow API

  • One factory function that generates several similar DAGs from a config

    One factory function that generates several similar DAGs from a config (in references/details.md)

    Pattern 2: Dynamic DAG generation

  • Conditional paths with BranchPythonOperator and trigger rules

    Conditional paths with BranchPythonOperator and trigger rules (in references/details.md)

    Pattern 3: Branching

  • FileSensor, S3KeySensor and ExternalTaskSensor examples

    FileSensor, S3KeySensor and ExternalTaskSensor examples (in references/details.md)

    Pattern 4: Sensors

  • Task and DAG failure callbacks, retries and trigger rules

    Task and DAG failure callbacks, retries and trigger rules (in references/details.md)

    Pattern 5: Error handling and alerts

  • pytest checks with DagBag for import errors, structure, dependencies and cycles

    pytest checks with DagBag for import errors, structure, dependencies and cycles (in references/details.md)

    Pattern 6: Testing DAGs

  • A suggested airflow/ folder layout with dags, plugins and tests

    A suggested airflow/ folder layout with dags, plugins and tests (in references/details.md)

    Project structure

From the official docs

Questions

What does the airflow-dag-patterns skill do?

It gives the assistant an Airflow playbook: DAG design principles, six patterns (TaskFlow, dynamic DAGs, branching, sensors, error handling, tests) and a project layout.

How do I install airflow-dag-patterns?

Use the install box at the top of this page. For Claude Code it lists the plugin route (/plugin marketplace add wshobson/agents, then /plugin install data-engineering@claude-code-workflows) and two skills-only commands: npx skills add wshobson/agents --skill airflow-dag-patterns and gh skill install wshobson/agents airflow-dag-patterns (GitHub CLI 2.90 or later), which skip the plugin's agents and commands.

Does it need an Airflow install or API key?

SKILL.md names no API key. The examples import Airflow classes, and the TaskFlow pattern is labelled Airflow 2.0+, so you need Airflow in your own project to run what the assistant writes. The sensor examples use S3 and local files, so those need the matching Airflow provider and connections.

Is it safe?

Its folder holds two Markdown files and no scripts. The examples are Python DAG code: the quick start sets email_on_failure to True, the TaskFlow and sensor examples use S3 paths and a bucket name (data-lake) as placeholders, and the alert callback has a commented-out Slack stub. Check connections, buckets and alert targets before you deploy a generated DAG.

Should I install the whole data-engineering plugin?

Only if you want the rest. The README documents a plugin route for Claude Code and Codex. The data-engineering plugin folder holds 2 agents, 2 commands and 4 skills, and its repo tree shows no hooks or MCP config files. The install box offers the plugin and the skills-only commands.

Does it work in Codex or Cursor?

The README documents Codex CLI as a plugin route and a skills-only install. For Cursor it points to a plugin marketplace and its setup docs, which we did not check, so follow the README there.

Is this skill from Anthropic?

No. It is in wshobson/agents, a community collection. The skill sits in the data-engineering plugin there, alongside data-quality-frameworks, dbt-transformation-patterns and spark-optimization.

Information

updated Nov 2, 2026

Repository
wshobson/agents
Folder
plugins/data-engineering/skills/airflow-dag-patterns

wshobson/agentsskills.sh

Sources

Docs, issue trackers and discussions this page is built from.

Show all 5
  1. airflow-dag-patterns SKILL.md · github.com
  2. wshobson/agents README · github.com
  3. data-engineering plugin.json · github.com
  4. wshobson/agents marketplace.json · github.com
  5. details.md · github.com

Updated Nov 2, 2026.