Embed images
Turns every image into a vector with a vision model (CLIP or DINOv2) on the CPU, an NVIDIA GPU or Tenstorrent cards; images per second live; optional zero-shot labels.
How it works
The model (CLIP ViT-B/32 or DINOv2 small, open weights from the Hugging Face Hub, downloaded once into the worker's cache) turns each image into a vector; images that look alike get vectors that point the same way. Find duplicates, Check labels and Embedding map use them.
Accelerator: auto takes the best the worker has (Tenstorrent cards through PyTorch/XLA for Tenstorrent, else an NVIDIA GPU in half precision, else the CPU) and says which in the log and the live data; naming one makes the block stop if it is not there, never run elsewhere. The images per second go out live (channel embed), for gauges that compare workers.
Prompts (CLIP only): labels to test, separated by commas, such as a zebra, an empty landscape; each image gets the one that fits it best, with no training (zero_shot.csv). Simple features needs no download: colour layout and edges, for tests on small workers.
Inputs
| Input | Takes | What it is |
|---|---|---|
Imagesimages | Files jpg, jpeg, png, webp, bmp, tif | Images, in folders (the folder is the label) Takes several connections. |
Outputs
| Output | Gives | What it is |
|---|---|---|
Embeddingsembeddings | Files npy, json | embeddings.npy (one row per image), paths.json (their paths), embedding.json (model, backend, images per second) |
Zero-shottable | Table csv | zero_shot.csv: the prompt that fits each image best, with the scores (with Prompts) |
Live telemetrytelemetry | Live telemetry | What the block sends while it runs, for views such as Chart or 3D view. |
Settings
| Setting | Default | What it does |
|---|---|---|
Modelmodel |
CLIP ViT-B/32 | One of: CLIP ViT-B/32, DINOv2 small, Simple features (no download) |
Acceleratoraccelerator |
auto | auto: Tenstorrent cards, else an NVIDIA GPU, else the CPU of the worker that runs it; naming one stops the block if it is not there One of: auto, CPU, NVIDIA GPU, Tenstorrent |
Batchbatch |
32 | Images per step From 1 to 512 |
Prompts (CLIP)prompts |
empty | Labels to test, separated by commas: a zebra, an empty landscape |
Where it runs
On one of your computers with the EsaSim worker: Any worker, NVIDIA GPU, Tenstorrent.
The worker installs the Python packages it needs when they are missing: torch[cuda]</code>, <code>transformers</code>, <code>pillow</code>, <code>numpy.
In its code, settings are ctx.params["key"], inputs ctx.inputs["id"] and outputs ctx.output("id", "name.ext"); see Custom blocks.