If you drop a folder of raw takes onto a coding agent and ask it to edit a video with ffmpeg, the first five minutes feel like magic. Exit code 0, file rendered, task closed. Then you hit play and the pain begins: a sentence is cut off mid-word, transitions jerk around, and in the tiny facecam bubble the speaker is looking away. The agent gets the edit 90% there, but fixing the last 10% of tiny glitches takes longer than editing by hand.
Developer Cole Medin ran into this himself and released the free, open-source edit-video skill for Claude Code, Cursor, Codex, and any agent that supports SKILL.md. An agent skill is a folder of instructions, scripts, and rules that teaches an agent a repeatable workflow without long prompts. Cole edited his own YouTube breakdown of the tool with it and says that for tutorials, presentations, and screencasts, he no longer needs a human editor.
Five Steps and Scripts Instead of Eyes and Ears
A text model physically cannot hear milliseconds on an audio track or see the frame at a cut point. Ask it to 'guess the timestamps' and it will miss.
In edit-video, the roles are flipped. The agent acts as the conductor, while 37+ deterministic Python scripts in the scripts/ folder do the precision work. Cole didn't write them as textbook code—his assistant agent grew these utilities for itself to serve as its 'eyes and ears'.
At the top of SKILL.md sits a format table. If you give very little context, the agent asks one clarifying question at step zero and assembles the right chain of steps for a talking-head video, a screencast with a face bubble, animated B-roll (supporting visuals over the speaker that illustrate a point and hide cuts), Shorts, or lectures. The basic five-step pipeline runs completely locally and for free on top of your agent subscription:
- Transcription and take filtering (
transcribe.py+align.py). Local Whisper provides the text, but the skill doesn't trust its raw timestamps. Forced alignment in PyTorch runs on top, pinning every word to exact milliseconds on the track so bad takes get filtered out. - Pause removal (
pauses.py,cutpoints.py). The scripts output a table of silences, breaths, and retakes, and the agent decides what to cut. - Assembly (
assemble.py,screen_bubble.py,insert_segment.py). Takes are stitched together, screen recording and camera are synced by audio, and pre-built segments are dropped in. - Polish (
transitions.py,captions.py, HyperFrames). Smooth transitions with sound effects, subtle punch-in zooms to hide jump cuts, captions, and programmatic animations are added. - Export and self-validation (
export_final.py,validate_cut.py,retranscribe.py). Here is the main safety net: a deterministic validator script re-transcribes the finished file and runs a word-by-word diff against the source audio. If a word was clipped or doubled, the agent fixes the cut itself.
All dependencies live in a single $S/.venv environment managed by uv (you need ffmpeg, Python, and Node for animations):
# Install the edit-video skill via the skills CLI
npx skills add coleam00/skills --skill edit-video
# Check ffmpeg, Python, and the uv environment with doctor.py
S="${CLAUDE_SKILL_DIR}/scripts"
uv run --project "$S" python "$S/doctor.py"
# Optionally install forced alignment for exact timestamps
uv sync --project "$S" --extra align --inexact
# Start editing right inside your Claude Code chat
/edit-video /path/to/raw_clips
The doctor.py script shows right away what is missing on your machine, and /edit-video with the path to your raw clips kicks off the loop. Your source folder can be a total mess—the agent picks out the usable takes and builds a rough cut from your short description.
Lessons from lessons.md and Palette in brand.json
Scripts cover the heavy routine, but video editing has plenty of rare traps that don't warrant a dedicated Python file. For those, references/lessons.md keeps a plain-English bug log: catch a render bug once, write down the rule, and the agent never repeats that mistake.
Cole's repo already has three telling lessons. First, exit code 0 proves nothing—the agent is forbidden from reporting success without inspecting actual frames. Second, you have to inspect crops of what actually matters: on a full contact sheet of preview frames, you can't see that the speaker looked away inside a 250 px face bubble, and three transitions passed the wide check only to fail on the crop. Third, always check the final export: once, inserting a sponsor segment shifted the audio after it by 17 ms, even though the intermediate file showed 0.0 ms.
Visual style lives in examples/brand.json, which stores background colors, accents, caption colors, and fonts for animations:
{
"bg": "#0B0E14",
"accent": "#4C8DF6",
"font": "Inter",
"mono": "JetBrains Mono",
"caption_ink": "#F4F6FA",
"caption_active": "#8AB6FF"
}
Just swap in your own colors and fonts—or on your first run, ask the agent to walk through brand.json and lessons.md with you to tailor the skill to your channel's style.
A Multimodal Pipeline for $2.71 and Honest Limits
Local Basic mode handles most rough cuts, yet a text LLM still has four blind spots: studio-grade transcription accuracy, watching the finished video, generating thumbnails, and surgical voice fixes.
For those, there is an extended Full mode via OpenRouter, a single gateway to multiple models under one API key. Full disclosure: Cole's video was sponsored by OpenRouter, though the skill itself is open-source, Basic mode works without keys, and Full mode only turns on when you explicitly ask for it.
Editing his entire video on Fable 5.2 through the full pipeline cost Cole $2.71 total. Here is what that stack caught and fixed:
| Task | Model via OpenRouter | What It Caught and Fixed in the Video |
|---|---|---|
| Precision transcription | elevenlabs/scribe-v2 (or microsoft/mai-transcribe-2 at half the price) | Spotted a tiny mid-word stumble and helped fix a clip cut off mid-word |
| Video review | Gemini 4 Argon (via 480p 10 fps proxy with audio) | Heard faint audio crackle caused by CPU overload during recording, which the agent then removed |
| Voice fix | Fish Audio 2.1 Pro (from an ~8-second voice sample) | For a couple of cents, replaced a spoken '70%' with '75%' to match the slide |
| Thumbnail drafts | Flux, Gemini 3 Pro, GPT Image 2 | Quickly generated cheap drafts using the speaker's face for idea selection |
Voice cloning comes with an honest trade-off: the fixed phrase sounds identical to the original, though on camera there is slight lip desync. For faceless segments it's a non-issue, and on a single word viewers rarely watch your lips—whereas a wrong number right next to a slide jumps out immediately. Thumbnails are also meant as drafts (models can mangle the Claude logo) to pull good ideas from. On top of that, long_to_shorts.py automatically cut three ready-to-post Shorts from the long video, including a 35-second clip with zoomed-in screen graphics.
Full mode is enabled explicitly, with per-call cost tracking:
# Set your OpenRouter key (or put it in scripts/.env)
export OPENROUTER_API_KEY="sk-or-..."
# Enable Full mode explicitly for the current project folder
S="${CLAUDE_SKILL_DIR}/scripts"
uv run --project "$S" python "$S/openrouter.py" ./ws enable
# Check exact costs for each model call
uv run --project "$S" python "$S/openrouter.py" ./ws costs
# Optionally run the word-by-word cut check manually
uv run --project "$S" python "$S/validate_cut.py" ./ws
You can store the key in scripts/.env—it is already in .gitignore so it won't leak into your repo. After that, just add 'use the full pipeline' in your chat with the agent, and it will turn on cloud models with cost logging.
- Footage and voice. Basic mode is 100% local. In Full mode, audio and compressed video go to external providers via OpenRouter, and
fix_word.pyasks for explicit speaker consent before sending a voice sample. For strict NDA material, stay in local mode. - Environment and format. The
.venvenvironment takes around 500 MB and rebuilds when you update the skill (fast from theuvcache, though flags like--extra alignmust be re-added). The skill excels at tutorials, presentations, and screencasts, but it won't replace Hollywood-style entertainment editing—and a final human watch before publishing is still mandatory.

