At 2:00 a.m., the problem doesn’t look like “AI video.” It looks like a production incident: a queue backing up, a generation job failing halfway through, a character drifting out of continuity, and a user waiting on a render that suddenly doesn’t make sense. At that moment, the difference between a demo and a product isn’t a model. It’s the system built around it.
That “operating system” is exactly what Jayesh Gaur has spent nearly two years building at Story.com. As a founding engineer, Gaur has been the primary architect behind the platform’s agentic orchestration layer: the system that plans, coordinates, validates, and assembles the many moving parts required to produce coherent long-form stories and movies from a single user intent.
His remit spans the ML-facing logic and the production backend, so the workflow ships as a product users can trust.
Below is a lightly edited conversation with Gaur, focused on what he owned, what was original, and why it mattered in production.
Q: In plain terms, what is the “director layer” for long-form AI movies?
A: Generative media has made remarkable progress at producing short, impressive outputs. The hard part is turning those outputs into something users can control, iterate on, and trust, especially when the goal is a longer narrative instead of a single clip.
In long-form storytelling, a single prompt can’t do the job. The system has to handle a chain of decisions: interpret intent and plan a story structure, maintain continuity across characters and scenes, generate assets across modalities like text, images, motion, and audio, assemble results into a coherent sequence, evaluate outputs and revise when they fail constraints, and do all of this quickly and cheaply enough that the workflow feels usable.
That’s what I mean by a director layer. It’s the application system around the models that makes results repeatable.
A visual overview of Story.com‘s orchestration pipeline specialized AI agents collaborate across planning, media generation, assembly, and delivery to produce coherent long-form video from a single user prompt.
Q: What is your role at Story.com and what responsibilities do you actually own?
A: I’m a founding engineer at Story.com, and I’ve spent nearly two years designing what I call the “operating system” for long-form generation. I’m the primary architect behind our agentic orchestration layer: the system that plans, coordinates, checks, and assembles the many moving parts required to produce coherent long-form stories and movies from a single user intent.
My responsibility spans both the ML-facing logic and the production backend, so the pipeline ships as a product users can trust. That includes how we break a project into stages, how agents hand off state without drifting, how we recover from partial failures, and how we keep the pipeline fast and cost-efficient at scale.
Q: Why does orchestration matter more than just calling a better model?
A: Better models help, but they don’t solve the product. Long-form workflows fail in ways that don’t show up in short demos: partial failures, state drift, compounding inconsistencies, and users wanting edits instead of restarts.
Orchestration is what turns probabilistic generation into a workflow. It’s how you break a long objective into steps, coordinate them, keep state consistent, detect failures, recover without restarting everything, and keep the experience fast and cost-efficient.
That’s why I say the moat isn’t a single model. It’s orchestration that makes results repeatable.
Q: When you say “multi-agent orchestration,” what does that mean in your system?
A: It means we break a long objective into coordinated sub-tasks, delegate them to specialized components, and use feedback loops to converge toward an acceptable output.
We don’t force one model to do everything. We decompose the workflow into roles for planning, continuity, asset generation, quality checks, and assembly, then coordinate them with state, constraints, and retries.
A simplified view looks like:
- Planner: translates user intent into story beats and scenes
- Continuity layer: tracks characters, setting, and narrative facts
- Asset generation: produces imagery, motion, and audio components
- Evaluator or QA: checks constraints like coherence, safety, and quality thresholds
- Assembler: composes the final narrative output and supports edits and regenerations
The architecture matters because every stage can fail differently. You need system-level controls: guardrails, rollback paths, and evaluation loops that allow partial regeneration without collapsing the whole output.
Q: What did you personally build that is original here?
A: Three things I personally owned and built end-to-end are:
First, the agentic orchestration layer that plans, coordinates, retries, and stitches outputs into a coherent long-form result.
Second, continuity and evaluation checks catch drift and quality issues early, so long-form generations stay consistent instead of slowly falling apart across scenes.
Third, production pipeline controls for scale: queues, observability, failure recovery, and cost and latency constraints, so the workflow stays usable as usage grows.
Q: What makes long-form unusually hard compared to short demos?
A: “Long-form isn’t hard because the idea is complicated. It’s hard because reality is messy.
Long-form generation introduces problems that don’t show up in short demos: state drift where a character’s traits change across scenes, compounding errors where small inconsistencies cascade over time, partial failures where one asset fails and breaks assembly, latency pressure where long jobs still need to feel interactive, cost pressure where compute must stay viable as usage scales, and editing expectations where users want targeted revision, not full restarts.
Solving those constraints requires systems design, not just model selection. It requires building a pipeline that can plan, inspect, revise, and finish while keeping the experience smooth.
In practice, the orchestration work lets us recover from partial failures, enforce continuity checks, and regenerate only what broke instead of restarting the whole story.
Q: How do you measure whether this is working in the real world?
A: You feel it immediately in product outcomes because real usage punishes brittle systems.
At scale, the goals become concrete: reduce failure rates, keep costs under control, and improve retention by making results consistently usable.
We also look at power-user behavior because it reflects whether the workflow is becoming repeatable. When someone is generating and iterating at high volume, it usually means the product is dependable enough to be a real creative tool, not just a novelty.
Q: Why does this matter for the broader industry?
A: A useful way to understand where this is going is that prompts are giving way to operating systems. The early wave rewarded clever prompting and isolated outputs. The next wave is about systems that coordinate many steps and produce results users can refine.
I think models evolve in two phases. First, capabilities improve. Then the real work is building applications that turn those capabilities into practical workflows and reduce manual effort.
If long-form AI movies become mainstream, it won’t be because a single model got better. It’ll be because the orchestration systems got good enough that the workflow feels natural: generate, edit, regenerate, and ship without fighting the underlying complexity.
