← Claude
Creator Magic
Vibe Coding an AI Shorts Editor (Remotion)
⏱ 56 min video · 3 min read12 Jun 2026Worth watching
TL;DR
A live stream where the creator vibe-codes a fully automated AI shorts editor from scratch using Claude Code, Remotion, FFmpeg, and Hyperframes — no SaaS, no templates. The tool ingests long-form YouTube videos, uses YouTube analytics retention data to find peak moments, transcribes on-device, generates an edit decision list, burns in captions, and outputs upload-ready short-form clips.
Key points
1
The creator builds a working AI shorts editor live using Claude Code (orchestrated via his own 'Tank' framework), Remotion for captions/graphics, FFmpeg for rendering, and Hyperframes for animated overlays — all without touching a traditional video editor.
2
YouTube Analytics retention CSVs are fed into the AI as signals to identify the most engaging moments in a long-form video, giving the AI context before it generates an edit decision list (EDL).
3
All transcription and AI inference runs locally on a Mac Studio M3 Ultra using models like Hermes 70B, GPT-O 120B, and Qwen 3 — with optional cloud fallback to 11 Labs Scribe (which supports 90+ languages with auto-detection) or OpenAI Whisper.
4
Claude Fable 5 (Claude 4 Opus-level 'Mythos' model) is highlighted as exceptional for architecture, security auditing, and planning — the creator found it autonomously audited his entire AWS account, flagged unused keys, and identified money leaking from forgotten S3 buckets.
5
The end-game vision is full autonomy: when a live stream ends in OBS, the pipeline automatically processes the footage, selects clips, adds animated captions and VFX, and publishes to YouTube Shorts, TikTok, and Instagram without human intervention.
Actionable insights
→
Export YouTube Analytics retention data as a CSV and feed it to your AI tool before asking it to select clip moments — it gives the model a real engagement signal rather than guessing.
→
To build your own shorts editor, prompt Claude Code (or Cursor/Codex) to use Remotion, Hyperframes, and FFmpeg, and tell it to produce edit decisions as JSON EDL files — no video editor knowledge required.
→
Run heavy transcript/inference tasks locally (e.g. Mac Studio M3 Ultra with MLX) to avoid API costs, and reserve frontier models like Claude Fable 5 for architecture, planning, and security audits where their reasoning pays off.
→
Use Claude Fable 5 to audit cloud infrastructure (e.g. AWS) — the creator found it proactively identified leaking costs and insecure keys beyond what he asked, potentially saving thousands annually.
→
Build manual click-through steps first when vibe coding a complex pipeline, then automate them later — this keeps you as the human-in-the-loop while the app is unstable and makes debugging far easier.
Notable quotes
“I am going from a bill that is $149 a month to a bill that is about 15 cents a month. I kid you not. Amazon SES is basically what all email list providers are using under the hood anyway.”
“If it saves me $149 a month on an email SaaS service, that is $1,788 a year. So I am quite happy to sling 20, 30, 40, 50 bucks on it.”
“The moment the 22nd of June comes around, I am going to be whipping out my credit card and I am going to hard lock it into Anthropic and I am going to be like: take my money, because it is worth it.”
Worth watching?
✅
Worth watching the full video?
Watch if you want a real-time walkthrough of building an AI-powered video automation pipeline — the key architecture, tools, and workflow decisions are all captured here, but the live stream footage shows the actual build breaking and iterating in ways the summary cannot fully convey.
Saved you some time? The creator still deserves a like.
Want this for your own channels?
Add the channels you follow. Every new video summarised and in your inbox the moment it drops. From £4/month.
Try it free