Choose based on your starting material
Text-to-video is useful when the whole scene is still flexible. Image-to-video is useful when a finished, authorized still already establishes the character, setting and framing.
Text-to-video
Describe one fictional adult subject, one main action, one camera movement and one environmental motion. Short clips become less coherent when a prompt contains too many simultaneous actions.
Image-to-video
Use a source you own or have permission to animate. Prompt mainly for movement, camera and atmosphere rather than redescribing every visual detail. Large motion may force the model to reconstruct more of the source.
Review frame consistency
Check the face, hands, body structure, background geometry and lighting from beginning to end. Video generation uses credits, so start with restrained motion before attempting complex direction.