Claude Video Vision: Let Claude Watch Videos

• Watches a video and returns frames and a transcript

  Takes a video file or YouTube link, pulls still frames with ffmpeg, and gets a timed transcript of the audio. Claude sees the frames as images and reads the words with timestamps. YouTube links are downloaded first.

  Try: “Watch ~/Downloads/demo.mp4 and tell me what the bug is”

  Runs code · video_watch

• Maps a long video before you look closer

  Checks the video's structure with ffmpeg: scene changes, silent stretches and motion. It can also transcribe the audio. It extracts no frames, so Claude can plan which parts to look at.

  Try: “Analyze this one-hour lecture and find where the topic changes”

  Runs code · video_analyze

• Looks closely at chosen parts of a video

  Pulls frames from the time ranges you pick, at the frame rate and size you choose for each. With the optional index on, it keeps frames cached so repeat looks are quick.

  Try: “Look closely at 1:25 to 1:35 and read the text on screen”

  Runs code · video_detail

• Shows a video's length, size and format

  Returns metadata such as duration, resolution and codec for a file or YouTube link, without processing it.

  Try: “How long is this video and what resolution is it?”

  Runs code · video_info

• Checks that the needed programs are installed

  Looks for ffmpeg, the Whisper engine, the Gemini key and the optional yt-dlp, and tells you what is missing and how to install it.

  Try: “Check whether video watching is set up correctly”

  Runs code · video_setup

• Changes video settings, and can clear saved sessions

  Sets the audio backend, Whisper model, frame size, frame rate and similar options in the config file. One option also deletes the saved session cache folder.

  Try: “Switch the audio backend to local Whisper and use 720-pixel frames”

  Delete · video_configure

Watch out for

• Gemini and OpenAI backends upload your audio

  With either cloud backend, the audio from your video goes to that company under your API key. Pick the local backend for private videos. github.com

• It runs programs on your computer

  The server starts ffmpeg, and yt-dlp and Whisper when you use them, and downloads Whisper models with curl. Install those tools from sources you trust. github.com

• One setting deletes the saved session cache

  The clear-sessions option in video_configure removes the sessions folder under ~/.claude-video-vision. It does not touch your videos. github.com

• YouTube downloads and models take disk space

  Downloaded videos and Whisper models are stored under ~/.claude-video-vision. Old downloads and sessions are cleaned up after 7 days by default. github.com

• The tool descriptions tell Claude how to order calls

  The watch tool says to call the analyze tool first for videos longer than 30 seconds. This changes how many calls Claude makes. github.com

Security advisories