Why AI video still can’t reproduce a real person — and what we are changing to get closer

We recently gave a state-of-the-art video model five clips of the same person and asked it to put her in a short campaign. She was, at that point, arguably the best-documented person in the whole project: five separate reference sources, a written description of her hair, her build, her jewellery, her accent. The model returned a different voice, a different face, and a different body type in nearly every shot.

This was not a vendor failure. It was the single hardest open problem in generative video, showing up exactly where it always shows up. This post is about why it happens, why “just give it more reference images” does not fix it, and what we are changing to make it happen less.

The problem is not quality

The clips were not bad. They were polished, well-lit, on-script. The problem was that they were different people. A dark-haired woman in the first shot, a different hairline and a different jaw in the second, a voice an octave and an accent away in the third. Any one of them, on its own, would have passed a casual glance. Side by side, they read as a casting error.

That gap between “each frame looks real” and “all the frames look like the same real person” is where the entire field currently lives. Diffusion models are extraordinary at plausibility and bad at persistence. They will draw you a convincing face every single time; they will not promise you the same face twice.

Why each shot is a new person

The root cause is architectural, not cosmetic. A video model has no memory between generations. Each 8-second shot is a completely independent sample from a probability distribution. When you ask for shot two, the model does not recall shot one — it rolls the dice again, conditioned only on whatever you handed it this time.

Google’s own Veo 3 technical report is unusually honest about the consequence: the model must “accurately depict the specific details mentioned in the prompt while filling in all of the underspecified aspects of the scene.” Every detail you do not specify is a detail the model is free to invent differently next time. And the number of details in a human face — the exact jaw angle, the distance between the eyes, the way the hair falls, the width of the shoulders — is effectively infinite. You cannot specify all of them.

Reference images remember the category, not the person

The intuitive fix is to hand the model a picture. “Here is the person. Now render her in this scene.” Reference-to-video models accept exactly that. The catch is what the reference actually does.

When you give a model a reference image, it does not do a gradient update. It does not fine-tune a LoRA. It does not build a persistent identity embedding that it will reuse across calls. It conditions this one generation on an image feature, the way it conditions on a text prompt — a loose, lossy summary of “a dark-haired woman in her late twenties,” not a mathematical commitment to a specific face.

So five seed clips do not produce five times the identity. They produce five slightly different summaries of the same category, and each shot samples a new member of that category. A phrase like “a confident woman” describes half the species; the model has nothing to hold, and it invents per shot. We learned this the expensive way on our own montage feature — a paid render came back with a dark-haired woman on the street and a blonde on the rooftop, from a byte-identical prompt and the same reference image on both shots.

The techniques that would fix it are not exposed

The research community has solved a lot of this, for images. IP-Adapter injects an image as a prompt through decoupled cross-attention. InstantID goes further, extracting a face embedding and conditioning an entire IdentityNet on it so the generated face is locked to the reference at a structural level. DreamBooth and character LoRAs go further still, fine-tuning the weights on your specific subject so the model genuinely learns that person.

Those are real fixes, but they are not what a consumer video API gives you. The big commercial models expose reference images and a prompt, not an ID embedding, not a LoRA slot, not a fine-tuned checkpoint. You are asked to reproduce a person with the two controls that are provably the weakest at reproducing a person. The gap between the published research and the shipping product is the entire competitive landscape right now.

Body type is the forgotten axis

Even when a face holds, the body usually does not. Height, build, shoulder width, how the person carries themselves — these are entangled with pose and wardrobe in a way a face is not. A model can hold a face while quietly re-rolling the body because “build” was never pinned to anything concrete. Our own prompts said “same face, hair and build throughout” for a long time, and “build” did almost no work, because “build” is not a fact. “Slim build, about 170 centimetres” is a fact.

Voice is worse

If identity drift is the hard problem, voice drift is the same problem with fewer tools. The text-to-video models that generate their own audio — Veo’s reference-to-video with native audio, for instance — do not take a voice reference at all. There is no speaker embedding in the request. The model synthesises speech the same way it synthesises a background: as another dimension it is free to sample. So a person who should sound like herself sounds like a different actress every clip, because there is no conditioning signal holding the voice still.

The honest state of voice cloning makes the point. ElevenLabs documents that a usable instant clone needs one to three minutes of clean audio, and a high-fidelity professional clone wants thirty minutes to three hours. A video model that generates its own speech is not doing any of that. It is not cloning. It is improvising, once per shot.

That distinction matters beyond the technology, because a real person’s voice is not a creative asset. “Generated speech seeded from her footage” is not “her speaking.” It is a synthetic voice wearing a resemblance, and using it to put words in a real person’s mouth — without explicit, separate consent — is both a legal exposure and the fastest way to lose a client’s trust. A real person’s likeness and voice each need their own explicit consent, separate from permission to use their footage.

The two honest paths

Once you see the mechanism, the options collapse to two, and pretending there is a third is how teams waste budget.

Path one: a real person should be filmed, not generated. If the goal is a specific named human — a founder, a practitioner, a real face your buyers will meet — the correct answer is real footage. Real footage has a real voice, a real body, a real person. No model can drift, because there is nothing to sample. Every generated substitute is a downgrade in both fidelity and trust, and it reintroduces a consent problem that did not exist before. We recommend this path first.

Path two: an avatar needs a locked persona, not a lucky prompt. When the presenter is a fictional character — which is the legitimate home of AI video — the fix is not “describe her better and hope.” It is to treat the person as data: a fingerprint of concrete facts that is pasted, byte-for-byte, into every shot so the model has the same words to hold every time.

What we are changing

We already had the first half of the answer: a continuity block, a single identity paragraph composed once and repeated verbatim across every shot. Verbatim matters — two paraphrases of the same outfit are two outfits to a diffusion model. What we were missing was the second half: the block was mostly binding to the reference image, and a reference image alone does not pin identity. So we built a persona fingerprint. It is in testing and is not yet part of the live product; real renders like the one below decide when it ships.

1. The persona fingerprint

A persona is a small, structured set of concrete facts about one person: face, hair, body type, voice, and jewellery. Each field is optional, and each is locked exactly as written. Instead of “same face, hair and build,” the prompt now carries specific, verifiable claims — oval face, dark arched eyebrows, fair to light-olive skin; dark shoulder-length hair with a side part; slender build; small silver stud earrings — followed by the instruction to keep that one face, body type and voice in every shot.

The rule the code enforces is simple and brutal: never invent what the record does not state. A field left empty is simply bound to the reference image. Writing “brunette” because it sounds specific fights the picture when the picture is a blonde, and that fight is the drift again. Concrete facts only, silence otherwise.

2. The voice lock

Because a text-to-video model generates its own audio with no voice-conditioning input, the only lever a montage has is words. So the persona carries a voice clause — warm mid register, clear English, natural pace — and that clause is repeated byte-identically in every shot. It is not voice cloning, and we will not pretend it is. It is the difference between telling the model nothing about the voice and telling it the same thing every time. The former guarantees a new voice per clip; the latter gives the model one stable target to aim at.

For a real person, the voice lock is deliberately not the answer. The answer is real footage. The voice lock exists for fictional avatars, where the voice is part of the character and consistency is the goal.

3. The proving render

Offline tests cannot tell you whether a face holds. A clean typecheck and a green suite have never once caught identity drift, because every test asserts the thing the code controls and the defect is in what the model infers. So we budget a real render before we believe any of this.

We ran a two-shot montage on staging with a persona fingerprint, at a cost of $2.40, and looked at the frames ourselves. The result: the same woman in both shots — the same dark shoulder-length hair, the same dark eyes, the same fair skin, the same slender build, the same stud earrings — matching the reference, where the pre-persona baseline had produced a different woman per shot. That is not a guarantee of future performance; it is one honest data point, which is exactly what a probabilistic model gives you and exactly why we keep paying for the proof instead of trusting the prompt.

The honest limitations

None of this makes identity persistence solved. It makes it less probabilistic, and that is a smaller claim than marketing would like.

Fine facial detail still smooths over — a generated face has the porcelain texture of a generated face, and no prompt restores pores. Wardrobe still has a mind of its own under dramatic relighting. Hands are still hands. And voice is the least verifiable of all of it: we can confirm the words were spoken, but only a human listening can confirm it sounds like the same person, and we do not pretend otherwise in a handoff.

The honest position, which we are happy to put in writing, is this: for a real person, film them. For a fictional character, lock the persona and inspect the frames. Anyone who tells you a video model can reliably reproduce a specific human being from a few clips is selling you the category, not the person.

Where this is going

The field is moving. Voice-conditioned video, identity embeddings exposed to end users, and cheap character fine-tuning are all visible on the horizon, and we expect to adopt them as the vendors expose them. Until then, the moat is not the model — everyone has the same models. The moat is the layer that tells the model exactly what to hold, in the same words, every time, and that knows when the correct answer is to stop generating and start filming.

That layer is the part we keep building.