Agentic Video Editing With HyperFrames and Remotion Skills

What "agentic video editing" means in 2026 — and what it doesn't

It doesn't mean typing a sentence into Sora, Veo, or Kling and getting a clip back. Those models generate pixels from a prompt, probabilistically. You can't open a diff and ask one to nudge a headline two pixels to the left — you re-roll the whole generation and hope the new version is close enough. That's AI video generation. It's a different job.

Agentic video editing is the inverse: the agent writes source — markup, components, timeline data — and a deterministic pipeline turns that source into frames. Render the same composition twice and you get the same bytes out, frame for frame. That determinism is the entire point. It's what lets an agent iterate on a video the way it iterates on a codebase: change a value, re-render, check the result, change another value — instead of re-rolling a generation and hoping.

HyperFrames and Remotion Skills both live in this second category. Neither one generates footage from nothing. They compose, animate, caption, and time things that already exist — text, data, images, and (increasingly) clips generated by an actual video model. The most common production pattern right now combines both categories: generate a few seconds of photorealistic footage from a diffusion model, then drop that clip into HyperFrames or Remotion as a video asset and let the agent handle the captions, lower thirds, transitions, and pacing around it.

The two tools, in one paragraph each

Both ship as installable agent skills — packaged instructions an agent loads before it starts writing code, distributed through the open npx skills add convention that works across Claude Code, Cursor, Codex, Gemini CLI, and a few other coding agents. You're not teaching the agent the framework from scratch in every conversation; you install the skill once per project, and the agent reads it before touching a file.

Installing HyperFrames as an agent skill

You need Node.js 22 or later and FFmpeg on your machine. Docker is optional, for byte-identical reproducible renders in CI.

npx skills add heygen-com/hyperframes

This opens a picker. Install the core set — every project needs all eight:

On top of the core set, the picker offers optional workflow skills — install whichever match your inputs: /product-launch-video, /website-to-video, /faceless-explainer, /pr-to-video, /embedded-captions, /graphic-overlays, /motion-graphics, and /remotion-to-hyperframes. To skip the picker entirely and install everything at once:

npx skills add heygen-com/hyperframes --all

If you'd rather scaffold a project by hand and hand it to your agent afterward, that path works too — hyperframes init installs the skills automatically:

npx hyperframes init my-video
cd my-video
npx hyperframes preview   # opens the studio, hot-reloads on save
npx hyperframes render --output output.mp4

That gives you a project with meta.json (duration, resolution, fps), index.html (the root composition), a compositions/ folder for sub-compositions, and an assets/ folder for media. npx hyperframes lint is worth running before every render — it catches overlapping clips and conflicting GSAP tweens before FFmpeg wastes time encoding a broken timeline.

Installing Remotion Skills as an agent skill

You need Node.js and a coding agent already installed — Claude Code, Codex, or OpenCode. Most coding agents require a paid subscription to their underlying model.

Starting fresh

npx create-video@latest

Pick the Blank template, say yes to TailwindCSS, and say yes to installing Skills when prompted. Then:

cd my-video
npm install
npm run dev   # Remotion Studio at localhost:3000

Open a second terminal in the same folder and start your agent there:

cd my-video
claude   # or: codex / opencode

You now have a live preview running in one window and an agent in the other that already knows the project's structure. Prompt it directly — "make a 15-second title card with a fade-in over a dark background" — and watch the preview update.

Adding skills to an existing project

npx skills add remotion-dev/skills

This drops a modular set of rule files into the project that teach the agent Remotion's actual mental model — a central entry point, individual compositions, and shared settings for resolution and frame rate — instead of letting it guess. One independent count put the package at roughly two dozen rule files and among the most-installed agent skills on the public skills registry, which tells you how many teams have already standardized on this exact setup.

The HyperFrames pipeline: seven steps, one artifact each

Every well-structured HyperFrames video — whether it starts from a website, a PDF, a CSV, or a blank page — flows through the same seven steps. Each step writes a named file that the next step depends on, so you and the agent always know what's done, what's next, and where the creative decisions actually live on disk.

1. Capture — Ingest and understand the source (a site, a doc, a dataset). The agent prints a summary before continuing, so you can correct course before it builds anything. Skip this if you're hand-authoring from a blank page.
2. Design — Write DESIGN.md: a brand cheat-sheet covering palette, fonts, and reusable components.
3. Script — Write SCRIPT.md, the narration. Scene durations are derived from how long the narration actually takes to say, not from a guess.
4. Storyboard — Write STORYBOARD.md: per-beat creative direction — mood, motion, transitions, and an audit of which assets each beat needs.
5. Voiceover — Synthesize narration and a word-level transcript that later powers caption timing. Skip this on a no-narration brand reel.
6. Build — Turn the storyboard into runnable HTML, one composition file per beat. For anything with three or more beats, it's worth spawning a focused sub-agent per beat — each one gets fresh context, just its storyboard section, and the asset paths it needs, which produces noticeably cleaner output than building every beat in one long-running conversation.
7. Validate — Render, then check the actual captured frames against the storyboard, not just whether the render command exited successfully.

Remotion doesn't enforce a named pipeline the same way. The equivalent structure is just standard React project conventions — an entry composition, individual scene components, shared config for fps and resolution — which the skill teaches the agent to respect. If you want HyperFrames' document-driven planning step inside a Remotion project, there's nothing stopping you from writing your own SCRIPT.md and STORYBOARD.md before prompting; the framework just won't ask for them.

A composition, built to the actual rules

Whether your agent writes this or you do, every working HyperFrames composition follows three non-negotiable rules. Here's a minimal one that obeys all three:

<div id="root" data-composition-id="lower-third"
     data-start="0" data-width="1920" data-height="1080">

  <!-- timed element: needs class="clip" + all three data-* attrs -->
  <div id="name-card" class="clip"
       data-start="2" data-duration="4" data-track-index="0"
       style="position:absolute; left:80px; bottom:120px;
              color:white; font-family:sans-serif;">
    <div style="font-size:38px; font-weight:600;">Outclaw</div>
    <div style="font-size:20px; opacity:0.75;">AI Tools, Weekly</div>
  </div>

  <script src="https://cdn.jsdelivr.net/npm/gsap@3/dist/gsap.min.js"></script>
  <script>
    // timeline MUST be created paused, and registered by composition id
    const tl = gsap.timeline({ paused: true });
    tl.from("#name-card", { opacity: 0, x: -30, duration: 0.6 }, 0);
    window.__timelines = window.__timelines || {};
    window.__timelines["lower-third"] = tl;
  </script>
</div>

• The root element carries data-composition-id, data-width, and data-height — without these the renderer doesn't know what it's rendering.
• Every timed element needs class="clip" plus data-start, data-duration, and data-track-index — without these the renderer treats the element as static background, present for the whole video.
• The GSAP timeline must be built with { paused: true } and registered on window.__timelines under the composition's ID — this is what lets the renderer scrub the animation frame by frame instead of letting it play on its own clock.

This is exactly the kind of pattern that /hyperframes-core exists to enforce automatically — an agent that's read the skill won't forget the third rule, because it's the one that's invisible in a live browser preview and only shows up as a broken render afterward.

Prompting Remotion instead of dragging a timeline

With the skill installed and the dev server running, you iterate by talking, not by scrubbing. A reasonable opening prompt for a 15-second explainer:

Create a 15-second composition called "FeatureIntro" at 1080x1920, 30fps.
Scene 1 (0-5s): headline "Ship video from a prompt" fades and slides up.
Scene 2 (5-12s): three feature bullets enter one at a time, staggered.
Scene 3 (12-15s): logo lockup with a subtle scale-in.
Use our existing Tailwind theme colors. Keep transitions under 400ms.

The agent generates a root composition, a scene component per beat, and registers durations against the project's shared fps config — the same structure a developer would write by hand, just typed by the agent instead. Iterating is a follow-up message, not a re-prompt from scratch: "make the headline bigger," "swap scene 2's stagger direction," "the logo should hold for an extra second." Each change is a small diff to a React component, which is exactly why teams that already think in pull requests tend to gravitate here.

Where each one actually breaks

Every comparison post sells the upside. Here's the part that determines whether you'll be debugging at 11pm.

Pick by signal, not by hype

Most teams default to whichever tool they saw first in their feed. Pick on these signals instead.

Can the two talk to each other?

More than the rivalry framing suggests. HyperFrames' own skill catalog ships an explicit /remotion-to-hyperframes workflow skill — a sign that "migrate an existing Remotion project over" is common enough to warrant first-party tooling, not a hypothetical edge case.

FFmpeg is the connective tissue underneath both. HyperFrames uses it directly to encode captured frames; Remotion output frequently gets piped through it for post-processing, transcoding, or stitching with clips from other sources. If you're building a pipeline that needs to combine HyperFrames-rendered segments with Remotion-rendered segments — or with raw footage from neither — FFmpeg is where that final assembly happens, regardless of which framework wrote the individual pieces.

The hybrid pattern worth actually building: generate a few seconds of photorealistic footage from a model like Kling, Veo, or Sora, then bring that clip into whichever framework your team already uses as a <video> asset. The agent handles everything around it — captions, lower thirds, transitions, pacing — while the generative model handles only the part it's actually good at: pixels that don't need to be deterministic.

Start here

One thing worth knowing before you commit: the skill format both tools use — npx skills add <org>/<repo> — isn't locked to Claude Code. The same install registers slash commands in Cursor, Codex, Gemini CLI, and a handful of other agents, so switching coding agents later doesn't mean re-learning either framework from scratch.

This space changes its install command, pricing tier, or feature set roughly monthly right now. Outclaw tracks exactly that kind of churn in AI tooling — worth a follow if you'd rather hear about the next breaking change than discover it mid-render.