Agent
Reasons over your data in a loop with the LLM you connect, using skills you allow.
How it works
Connect an LLM and the data to work on (files, a table of items, logs). For each item, the agent reads it with its skills (read a file, look at an image, read a log), thinks, and records a decision: a label, a note, keep or reject. Edit the prompts in the settings; open the code to change the harness itself. Limits keep every item within a number of steps and tokens.
Inputs
| Input | Takes | What it is |
|---|---|---|
LLMllm | Model | |
Datadata | Anything | Takes several connections. |
Outputs
| Output | Gives | What it is |
|---|---|---|
Decisionsdecisions | Table csv | decisions.csv: one row per item with its label, confidence and reasoning |
Reportreport | Text md | |
Totalsresult | Metrics json | |
Live telemetrytelemetry | Live telemetry | What the block sends while it runs, for views such as Chart or 3D view. |
Settings
| Setting | Default | What it does |
|---|---|---|
System promptsystem_prompt |
You are a careful reviewer of robotics data. Look at each item, use the tools to inspect … | |
Task for each itemtask_prompt |
Item: {name} · Decide whether this item shows a failure worth turning into a simulation s… | |
Allowed labelslabels |
keep, reject | |
Skillsskills |
read_text, view_image, list_files, decide | Comma-separated: read_text, view_image, read_table, list_files, note, decide |
At most (items)max_items |
50 | From 1 to 100000 |
Steps per itemmax_steps |
6 | From 1 to 50 |
Token budget for the runmax_tokens_total |
200000 | From 1000 |
Where it runs
On one of your computers with the EsaSim worker: Any worker.
In its code, settings are ctx.params["key"], inputs ctx.inputs["id"] and outputs ctx.output("id", "name.ext"); see Custom blocks.
More in AI
Check labelsFinds images whose label is likely wrong, from the labels of the images that look most like them, and images unlike any other.Embed imagesTurns every image into a vector with a vision model (CLIP or DINOv2) on the CPU, an NVIDIA GPU or Tenstorrent cards; images per second live; optional zero-shot labels.LLMA language model for the Agent: an EsaSim model, a provider with your key, or your own server.Text to imageTurns a prompt into any number of images with a diffusion model downloaded from Hugging Face onto the worker, from small (Segmind Vega, 0.7B) to large (FLUX.1 schnell, 12B): one image per seed, on an NVIDIA GPU, a Tenstorrent card or the CPU, each shown live as it is made.Text to videoTurns prompts into short videos (up to 10 seconds) with an open text-to-video model downloaded from Hugging Face onto the worker, from small (Wan 2.1, 1.3B) to the best open ones (Wan 2.2 A14B, and LTX-2.3 and LTX-2.5, which also make the sound): several variants of each prompt, one per seed, on the worker's NVIDIA GPU, each streamed live, whole, as soon as it is made.