I had eight gorgeous 1080p clips and no clean way to turn them into one deliverable.
That was the moment the novelty wore off. The model gave me cinematic shots in about two minutes each. Then I sat there with eight separate files, mismatched loudness, one clip at the wrong frame rate, and a deadline. The fun part was over. The actual work was a folder of disconnected assets and a manual stitching job I didn’t want to do twice.
So I did what we do with any messy, non-deterministic output: I wrapped it in a pipeline.
This post is the pipeline. It is the prompt structure I use to get reproducible shots, and the ffmpeg script I use to turn a pile of raw clips into one normalized, captioned, archived file. None of it is exotic. That is the point.
AI video tools generate clips. They don’t assemble them. The engineering work isn’t in the generation, it’s in taming the output: getting consistent inputs, then normalizing, concatenating, and archiving the results so the process is repeatable instead of a one-off you babysit.
If you only take one thing: treat generated clips as untrusted build artifacts. Standardize them on the way in, and never hand-edit the assembly step.
I generate with Seedance 2.0, ByteDance’s text-to-video and image-to-video model. The model is genuine ByteDance tech (see the Seed team page), but the hosted web front-end I use doesn’t expose a public REST API, and most consumer-facing wrappers for it don’t either.
That sounds like a dealbreaker for automation. It isn’t. It just moves the automation to both ends of the manual step:
1. Before generation: a prompt template that produces consistent, reproducible shots, so I’m not improvising every time.
2. After generation: an ffmpeg pipeline that ingests whatever lands in a folder and produces one clean output.
The human paste-and-click in the middle stays manual. Everything around it doesn’t have to be.
My first batch was bad. Stiff motion, a product that morphed between frames, audio that drifted out of sync. The problem wasn’t the model. It was that I wrote prompts like a person describing a vibe instead of a director naming a shot.
The fix was a fixed slot structure. Vague in, vague out:
[subject + setting], [what moves and how], [lighting], [camera move over N seconds], [mood/style]
Concrete example, written to that template:
A matte-black water bottle on a wet concrete ledge, light rain,
slow droplets sliding down the surface, soft overcast key light from
the left, camera pushes in slowly over 4 seconds, calm premium mood.
The difference between that and “cool product video of a water bottle” is the difference between a usable push-in on the first try and three wasted generations. I keep these as plain text files, one per shot, named shot-01.txt through shot-08.txt, so a sequence is version-controlled and re-runnable by a human in minutes.
# A sequence is just a directory of prompt files + the clips they produced
shots/
├── shot-01.txt # prompt
├── shot-01.mp4 # generated clip
├── shot-02.txt
├── shot-02.mp4
└── ...
Now the inputs are predictable. Time to make the outputs predictable too.
Here is the mistake everyone makes: they try to concatenate clips directly and get audio drift, frame-rate jumps, or a hard fail. ffmpeg’s concat demuxer needs every input to share the same codec, resolution, frame rate, and audio layout. Generated clips rarely do.
So I re-encode every clip to one spec first. Before this step, roughly 7 of every 10 batches I tried to stitch failed or came out with audio drift. After it, almost none did. The concat demuxer stops fighting you once every input matches:
#!/usr/bin/env bash
# normalize.sh — bring every clip to one spec before stitching
set -euo pipefail
mkdir -p normalized
for f in shots/*.mp4; do
name=$(basename "$f")
ffmpeg -y -i "$f" \
-vf "scale=1920:1080:force_original_aspect_ratio=decrease,pad=1920:1080:(ow-iw)/2:(oh-ih)/2,fps=30" \
-c:v libx264 -preset medium -crf 18 \
-c:a aac -ar 48000 -ac 2 \
"normalized/$name"
done
What each part buys you:
scale=...force_original_aspect_ratio=decrease plus pad fits any odd resolution into a 1080p frame without stretching faces.fps=30 forces one frame rate, the thing that silently breaks concatenation.-ar 48000 -ac 2 standardizes audio to 48kHz stereo, so loudness and sync stop fighting you.-crf 18 keeps it visually clean; raise it to 23 for smaller files.With every clip on the same spec, the concat demuxer just works. I generate the file list from whatever is in the folder, so adding a ninth shot means dropping in shot-09.mp4 and rerunning, not editing anything.
#!/usr/bin/env bash
# assemble.sh — stitch normalized clips into one deliverable
set -euo pipefail
# Build the concat list in sorted order
: > list.txt
for f in $(ls normalized/*.mp4 | sort); do
echo "file '$PWD/$f'" >> list.txt
done
# Concatenate without re-encoding (fast, lossless, same spec)
ffmpeg -y -f concat -safe 0 -i list.txt -c copy stitched.mp4
# Optional: burn in captions from a subtitle file
if [ -f captions.srt ]; then
ffmpeg -y -i stitched.mp4 \
-vf "subtitles=captions.srt:force_style='FontSize=22,MarginV=40'" \
-c:a copy final.mp4
else
cp stitched.mp4 final.mp4
fi
# Archive with a timestamped, content-addressable name
stamp=$(date +%Y%m%d-%H%M%S)
hash=$(md5 -q final.mp4 2>/dev/null || md5sum final.mp4 | cut -d' ' -f1)
mkdir -p deliverables
cp final.mp4 "deliverables/${stamp}-${hash:0:8}.mp4"
echo "Archived deliverables/${stamp}-${hash:0:8}.mp4"
Because the concat step is -c copy, it’s near-instant and lossless. All the heavy lifting happened in the normalize pass, where it belongs. The archive name carries a timestamp and a hash prefix, so I can always trace a delivered file back to the exact run that made it.
One honest warning. The tool’s multi-reference mode (feeding text plus several images plus a reference clip in one shot) is the impressive feature, and it’s roughly twice the credit cost of a plain text-to-video generation. I learned that by burning through a free allotment in an afternoon, regenerating the same sequence because I kept tweaking one input.
The fix that saved me: nail each input down one at a time in the cheap single-reference mode first, then switch to multi-reference only for the final pass. Treat the expensive mode like a production build, not a dev loop.
I want to be straight about the limits, because a pipeline is only as trustworthy as its honesty:
I’ve been running this on clips from SeedAIVideo, a hosted front-end for Seedance 2.0 that I use for generation. Disclosure: it’s the tool I happen to generate with, and the pipeline above is deliberately tool-agnostic. The ffmpeg half works on output from any text-to-video model. If you use a different generator, only Step 1 changes.
The interesting engineering in AI video is not the model. It is the boring, reliable layer around it: standardized inputs, normalized artifacts, a concat step you never touch by hand, and an archive you can audit. Build that once and a folder of disconnected clips becomes a deliverable on rerun, not a manual job you dread repeating.
The full ffmpeg reference for the filters above is worth a read if you want to tune the normalize pass.
What part of your AI-asset workflow is still a manual step you babysit? That is usually the next thing worth scripting.
