AI video went from novelty to usable faster than almost anyone expected. It is still the most limited of the AI media types, and knowing exactly where the limits are saves a lot of wasted generation.
The three kinds
| Kind | What you give it | Best for |
|---|---|---|
| Text to video | A written description | Short standalone clips, B-roll |
| Image to video | A still image + motion prompt | Far more control, consistent look |
| Avatar / presenter | A script | Talking-head explainers, training |
These get lumped together and are genuinely different products. Picking the wrong one is the most common reason people conclude AI video "doesn't work."
Image to video is the one people underuse
If you want control, start from a still.
Generate or choose an image you are happy with, then describe only the motion — "slow push in", "the fabric moves in the wind", "she turns to look at the camera." You are giving the model the entire look, so it only has to solve movement.
Text to video asks it to invent composition, lighting, subject and motion all at once, and it will make choices you did not want. Two-step is slower and lands far more often.
What the limits actually are
Length. Clips are short — typically seconds, not minutes. Longer videos are assembled from multiple generations, and the joins are your problem.
Consistency between clips. The same character across two generations is genuinely hard. This is the single biggest constraint on narrative work.
Physics. Hands, feet touching ground, objects being picked up, liquids, anything with contact. Watch for feet sliding — the classic tell.
Text. Signage and captions in generated video remain unreliable. Add text in an editor afterwards.
Speed and cost. Video generation is far slower and dearer than images. A failed generation costs real money and several minutes, which changes how you should prompt.
Prompting for motion
Because generation is expensive, the discipline matters more than with images.
Describe the camera separately from the subject. "Static shot, subject turns left" is clearer than one blended sentence.
One motion per clip. Multiple actions in one generation is the fastest way to waste a run.
Specify pace. "Slow", "gradual", "gentle" — models default to more movement than you usually want.
Name the shot type. Wide, medium, close-up. Free, and it removes a whole category of wrong output.
Storyboard as stills first. Get the images right, then animate. Much cheaper than iterating on video.
What it is genuinely good for now
- B-roll and backgrounds — atmospheric footage behind a voiceover
- Social clips — short by nature, which suits the length limit
- Product motion — a still product shot given subtle life
- Concept and pitch work — showing an idea before committing a real shoot
- Talking-head explainers — via avatar tools, where the format is forgiving
What it is not ready for
- Anything longer than a minute with narrative continuity
- A recognisable person appearing consistently across many shots
- Precise choreography or timing to a beat
- Dialogue with accurate lip sync outside dedicated avatar tools
- Anything where a physics artefact would be embarrassing
The part people forget: audio
Most video generators produce silent clips, or audio you will replace. Budget for sound — music, effects, voiceover — as a separate step.
A well-cut silent clip with good audio reads as professional. A perfect clip with wrong or absent audio reads as AI.
Common questions
What is the best AI video generator? Depends on the kind. Avatar tools for talking heads, image-to-video for controlled cinematic clips, text-to-video for quick standalone shots.
How long can AI videos be? Short — seconds per generation. Longer pieces are assembled from multiple clips.
Can AI video generators make realistic people? Increasingly, in short clips. Keeping the same person consistent across several clips is the remaining hard problem.
Is AI video free? Free tiers exist with heavy limits. Video is expensive to generate, so free allowances are much tighter than for images or text.
Why does my AI video look wrong? Usually physics — feet sliding, hands passing through objects, or unnatural weight. Shorter clips and simpler motion reduce it.
Should I use text-to-video or image-to-video? Image-to-video for control, which is most of the time. Text-to-video when you want something quick and are relaxed about the specifics.
Does AI video come with sound? Usually not, or not usable sound. Plan to add audio separately.