Esacode

Docs / Blocks / AI

Text to video

Turns prompts into short videos (up to 10 seconds) with an open text-to-video model downloaded from Hugging Face onto the worker, from small (Wan 2.1, 1.3B) to the best open ones (Wan 2.2 A14B, and LTX-2.3 and LTX-2.5, which also make the sound): several variants of each prompt, one per seed, on the worker's NVIDIA GPU, each streamed live, whole, as soon as it is made.

Runs on a worker (Python) AI Version 1.2 Code you can open and adapt

How it works

Write one prompt per line (what happens, the camera, the light, the style) and how many videos to make of each: each variant starts from its own seed, so one prompt gives as many different videos as asked, and the same seed gives the same video again. A table of prompts (a prompt column) or a text file (one per line) can be connected instead.

Every video lasts at most 10 seconds (Seconds: 0 keeps the model's own length, 5 s for most). The model is downloaded the first time into the worker's EsaSim cache (only what the pipeline needs), then read from there.

Models: Wan 2.1 T2V 1.3B (480p, Apache-2.0, small: runs on an 8 GB GPU), Kandinsky 5.0 Video Lite 2B distilled (768x512, 16 steps, 5 or 10 seconds, MIT), Wan 2.2 TI2V 5B (720p, Apache-2.0, the default: a 24 GB GPU; slowly on 8 GB), HunyuanVideo 1.5 8.3B (480p, Tencent Hunyuan Community licence: not for the EU, the UK or South Korea; 14 GB and more), LTX-2.3 19B (768x512 with sound, 30 steps), Wan 2.2 T2V A14B (720p, Apache-2.0) and LTX-2.5 19B distilled (960x544 with sound, 8 steps, the best; gated: needs a Hugging Face token): these three want a cloud GPU with 80 GB and a computer with about 96 GB of memory.

LTX-2.3 and LTX-2.5 are under the LTX-2 Community licence (free for companies under 10 million US dollars of yearly revenue) and put their sound in the MP4. A model larger than the GPU still runs: its parts move between the computer's memory and the GPU as they are needed (slower), and the block says so.

Or any text-to-video model in the diffusers format on Hugging Face (owner/name) or in a folder of the worker, whose licence the block reads and records.

Every video is saved as an MP4 (H.264, small enough to open in the dashboard) with a still of its middle frame, and streamed live, whole, as soon as it is made: connect the block's live telemetry straight to a Video review view, which plays each video in full while the next one is being made (Stream live: stills only sends just the stills); videos.csv lists seed, prompt, length, size, steps, sound, time, model and licence; summary.csv gives one row per run that tables from several models join into a comparison; generation.json keeps the settings to make the same videos again.

The videos are not filtered: what they show is the prompt's doing; check the model's licence before using them.

Inputs

InputTakesWhat it is
Prompts
prompts
Table or Files
csv, txt, json
Optional: a table with a prompt column, or a text file with one prompt per line; used instead of the Prompts setting Optional. Takes several connections.

Outputs

OutputGivesWhat it is
Videos
videos
Files
mp4, jpg, json
video_01_seed1.mp4 … (with sound from LTX-2.x) and its still video_01_seed1.jpg; with several prompts video_01_p1_seed1.mp4 …; generation.json (model, licence, settings, time)
Videos table
table
Table
csv
videos.csv: n, file, still, prompt_n, prompt, variant, seed, seconds, frames, fps, width, height, steps, guidance, sound, made_in_s, mb, model, licence, device
Summary
summary
Table
csv
summary.csv, one row: model, repo, parameters_b, size, seconds, frames, fps, steps, device, memory, videos, seconds_per_video, first_video_s, load_s, total_s, licence
Live telemetry
telemetry
Live telemetryWhat the block sends while it runs, for views such as Chart or 3D view.

Settings

SettingDefaultWhat it does
Prompts (one per line)
prompt
A red fox trots through fresh snow in a birch forest at dawn, its breath steaming in the … What the videos show: the subject and what it does, the camera, the light, the style. Several lines: several prompts, each made Videos per prompt times
Videos per prompt
count
4 How many variants of each prompt: variant k uses seed First seed + k − 1
From 1 to 1000
Seconds
seconds
0 Length of each video, at most 10 s. 0: the model's own (5 s). Longer than the model was trained for (Wan: 5 s) moves less well
From 0 to 10
Model
model
Wan 2.2 TI2V · 5B · 720p (default, GPU 24 GB) From small to large, all downloaded from Hugging Face the first time: Wan 2.1 1.3B (Apache-2.0), Kandinsky 5.0 Video Lite 2B (MIT), Wan 2.2 TI2V 5B (Apache-2.0, the default), HunyuanVideo 1.5 8.3B (Tencent Hunyuan Community: not for the EU, the UK, South Korea), LTX-2.3 19B with sound, Wan 2.2 A14B (Apache-2.0), LTX-2.5 19B with sound (the best; gated). LTX-2.x: LTX-2 Community licence, free under 10 million US dollars of yearly revenue. The GPU sizes are those that hold the model whole; smaller GPUs work too, more slowly, with enough computer memory
One of: Wan 2.1 · 1.3B · 480p (small, GPU 8 GB), Kandinsky 5.0 Video Lite · 2B · 16 steps, 5 or 10 s (GPU 12 GB), Wan 2.2 TI2V · 5B · 720p (default, GPU 24 GB), HunyuanVideo 1.5 · 8.3B · 480p (GPU 14 GB, licence), LTX-2.3 · 19B · with sound · 768x512 (GPU 80 GB, licence), Wan 2.2 T2V · A14B · 720p (GPU 80 GB), LTX-2.5 · 19B · with sound · 960x544 (best, GPU 80 GB, token, licence), Other (Hugging Face id or folder below)
Other model
model_id
empty With Model: Other: a diffusers text-to-video model on Hugging Face (owner/name) or a folder on the worker. Check its licence
Negative prompt
negative
empty What the videos should not show. Empty: the model's own (for Wan and LTX-2: blurred, overexposed, static, deformed, extra limbs…); distilled models (Kandinsky Lite, LTX-2.5) ignore it unless Guidance is above 1
First seed
seed
1 Variant k of each prompt uses seed + k − 1; the same seed gives the same video. 0: different videos every run
From 0 to 2147483647
Steps
steps
0 0: the model's own (LTX-2.5: 8, Kandinsky Lite: 16, Wan 2.1 1.3B and LTX-2.3: 30, the others: 40 to 50). Fewer: faster, rougher. On LTX-2.5 any other number replaces its own 8-step schedule and makes it worse
From 0 to 100
Guidance
guidance
0 How closely to follow the prompt. 0: the model's own (1, that is off, for the distilled Kandinsky Lite and LTX-2.5)
From 0 to 20
Width (px)
width
0 0: the model's size (Wan 2.1: 832, Kandinsky and LTX-2.3: 768, LTX-2.5: 960, Wan 2.2: 1280). Rounded to what the model takes
From 0 to 1920
Height (px)
height
0 0: the model's size (Wan 2.1: 480, Kandinsky and LTX-2.3: 512, LTX-2.5: 544, Wan 2.2: 704 or 720). For a vertical video, swap the two
From 0 to 1920
File size
quality
small (opens in the dashboard) small: each MP4 kept under 4.5 MB, so it is uploaded to EsaSim and plays in the dashboard; high: finer, larger files, which stay on the worker and are fetched when opened
One of: small (opens in the dashboard), high (larger files)
Stream live
stream
whole videos whole videos: each video is sent live, whole, as soon as it is made, so the Video review view plays it while the next one is being made (connect this block's live telemetry straight to the view); stills only: just a still of each, as the files come at the end
One of: whole videos, stills only
Accelerator
accelerator
auto auto or NVIDIA GPU: the run goes only to a worker with an NVIDIA GPU, and waits while none is online. CPU: any worker, only for a test: a video takes hours
One of: auto, NVIDIA GPU, CPU (hours per video)
Hugging Face token
token
pasted in the editor, kept encrypted Empty: the Hugging Face token in the profile of whoever runs the flow. Gives higher download limits; needed for gated models

Where it runs

On one of your computers with the EsaSim worker: NVIDIA GPU, Any worker. The worker installs the Python packages it needs when they are missing: torch[cuda]</code>, <code>diffusers>=0.42.1</code>, <code>transformers>=5.21</code>, <code>accelerate</code>, <code>safetensors</code>, <code>sentencepiece</code>, <code>protobuf</code>, <code>ftfy</code>, <code>huggingface_hub</code>, <code>imageio-ffmpeg</code>, <code>psutil</code>, <code>pillow</code>, <code>numpy.

In its code, settings are ctx.params["key"], inputs ctx.inputs["id"] and outputs ctx.output("id", "name.ext"); see Custom blocks.

More in AI