Esacode

Docs / Blocks / AI

Embed images

Turns every image into a vector with a vision model (CLIP or DINOv2) on the CPU, an NVIDIA GPU or Tenstorrent cards; images per second live; optional zero-shot labels.

Runs on a worker (Python) AI Version 1.2 Code you can open and adapt

How it works

The model (CLIP ViT-B/32 or DINOv2 small, open weights from the Hugging Face Hub, downloaded once into the worker's cache) turns each image into a vector; images that look alike get vectors that point the same way. Find duplicates, Check labels and Embedding map use them.

Accelerator: auto takes the best the worker has (Tenstorrent cards through PyTorch/XLA for Tenstorrent, else an NVIDIA GPU in half precision, else the CPU) and says which in the log and the live data; naming one makes the block stop if it is not there, never run elsewhere. The images per second go out live (channel embed), for gauges that compare workers.

Prompts (CLIP only): labels to test, separated by commas, such as a zebra, an empty landscape; each image gets the one that fits it best, with no training (zero_shot.csv). Simple features needs no download: colour layout and edges, for tests on small workers.

Inputs

InputTakesWhat it is
Images
images
Files
jpg, jpeg, png, webp, bmp, tif
Images, in folders (the folder is the label) Takes several connections.

Outputs

OutputGivesWhat it is
Embeddings
embeddings
Files
npy, json
embeddings.npy (one row per image), paths.json (their paths), embedding.json (model, backend, images per second)
Zero-shot
table
Table
csv
zero_shot.csv: the prompt that fits each image best, with the scores (with Prompts)
Live telemetry
telemetry
Live telemetryWhat the block sends while it runs, for views such as Chart or 3D view.

Settings

SettingDefaultWhat it does
Model
model
CLIP ViT-B/32
One of: CLIP ViT-B/32, DINOv2 small, Simple features (no download)
Accelerator
accelerator
auto auto: Tenstorrent cards, else an NVIDIA GPU, else the CPU of the worker that runs it; naming one stops the block if it is not there
One of: auto, CPU, NVIDIA GPU, Tenstorrent
Batch
batch
32 Images per step
From 1 to 512
Prompts (CLIP)
prompts
empty Labels to test, separated by commas: a zebra, an empty landscape

Where it runs

On one of your computers with the EsaSim worker: Any worker, NVIDIA GPU, Tenstorrent. The worker installs the Python packages it needs when they are missing: torch[cuda]</code>, <code>transformers</code>, <code>pillow</code>, <code>numpy.

In its code, settings are ctx.params["key"], inputs ctx.inputs["id"] and outputs ctx.output("id", "name.ext"); see Custom blocks.

More in AI