Esacode

Docs / Build

Build your own blocks

Make a block of your own in Python (runs on your workers) or JavaScript (a view in the browser): its ports and their formats, its settings, its code, versions, sharing with the EsaSim library and copies on GitHub.

New to writing blocks? Tutorial: blocks that work together builds a worker block and views step by step, with live values, a button that talks back to a running block, and blocks on two workers. A project in code writes a block and its blueprint as files in your own editor, with git.

What a block is

Every block in the block menu is a definition plus code. Nothing about a block is built into the editor: the same rules apply to EsaSim's own blocks, to the blocks of the EsaSim library and to yours.

PartWhat it holds
Definition (block.json)Name, category in the menu, icon, one-line summary, help, inputs and outputs (ports), settings, and for worker blocks the hardware it can run on.
CodeA worker block is Python (backend.py) with def run(ctx); it runs on your workers, as a task of each run. A view is JavaScript (frontend.js) with function render(ctx); it draws in the browser, on the flow's View page and next to results.

A block of your workspace is in the block menu of every project of the workspace, under its category, and keeps a numbered version each time it is saved. Runs use its latest version.

Make one

  1. Open Projects > Blocks and choose New worker block or New view. In a flow, Add block > Make a block opens the same builder, and Save as my block in a block's settings turns a block you changed in a flow into a block of the workspace.
  2. Start from Hello world, or from a copy of a built-in block (on the Blocks page: Start a block from a copy). Its ports, settings and working code fill the form.
  3. Name it, choose its category and icon, and say in one line what it does.
  4. Add its inputs, outputs and settings (below). Until you change the code, the starter code follows the form.
  5. Write the code. Verify checks the syntax (with the same parser that colours it) and that run(ctx) or render(ctx) is there.
  6. Save block. The block page shows its definition, its code, the flows that use it and its versions; Download gives a zip with block.json, the code and a README.

The code panel and the builder can be widened: drag the left edge of the code panel, or the line between the form and the code in the builder, and use the full-screen button for the whole window.

Ports: inputs and outputs

A connection goes from an output of one block to an input of another. Each port has an id (a short name used in the code: a letter, then letters, digits or _), a label, a type, and optionally the formats it reads or makes and a line on what it carries. Inputs can take several connections and be optional.

TypeWhat travelsTypical formats
FilesAny files: the block receives a list of files (path, name, size).png, jpg, mp4, mcap, bin …
TableTables: CSV with a header row (also TSV, or a JSON list of objects). Views get the rows; worker blocks get the files.csv, tsv, json
ItemsRecords as JSON lines: one object per line, such as the decisions of an Agent.jsonl
3D sceneA scene as JSON in the format esasim.scene/1 (entities and runs of timed positions), with the 3D files it names.json, glb, gltf, obj, stl, ply
MetricsNamed numbers in one JSON object, such as {"success_rate": 0.69}.json
TextText or Markdown: reports, notes, logs.md, txt, log
ModelA connection to an AI model, from an LLM block (inputs only): ctx.llm.—
AnythingAn input that accepts files of any type.—
Live values (stream)Inputs of views only: the live values of a worker block (its Live telemetry output wired straight in) or of a Telemetry block. The view reads them in ctx.input("port").stream.—
Live telemetryValues a worker block sends while it runs. Every worker block has this output; you do not add it.—

A table, metrics, a 3D scene and text are files too: they can go into an input of type Files. When both ends list formats and none is in common, the editor warns (for example, a block making csv wired into an input reading png, jpg).

Python packages

A worker block can use any Python package. Before it runs, the worker installs the ones that are missing on that computer, with pip, into its data folder (python-packages, one set per version of Python, and pip's download cache in pip-cache), never into the system's Python or your user profile; later runs reuse them. Which ones:

  • those the code imports, found by themselves: import numpy installs numpy, from PIL import Image installs Pillow, import cv2 installs opencv-python-headless, from numba import cuda installs numba and numba-cuda[cu12] (the CUDA target with its CUDA libraries);
  • those listed under Python packages in the block's definition, for names that differ from the import, extras or versions: torch>=2.3, numba-cuda[cu12], pillow>=10,<12. Only names, extras and versions: no pip options or URLs.

The first run on a worker shows the installation in the log and takes as long as the download. The worker's Install the Python packages blocks need setting (on by default) can turn it off; the block then reports which packages are missing. GPU code needs the GPU driver on the worker; CUDA libraries come with packages such as numba-cuda[cu12] or cupy-cuda12x, so the CUDA toolkit is not needed. Blocks run in Docker install their packages in the container at each run, unless the image already has them.

PyTorch on an NVIDIA GPU. List torch[cuda] (with a version if you need one: torch[cuda]>=2.4) when the block must use the GPU. On Windows, pip's ordinary PyTorch has no CUDA, so the worker installs PyTorch built for CUDA 12.6 from PyTorch's own package index instead (about 2.5 GB, once per computer) and replaces a PyTorch without CUDA it installed before. On Linux the ordinary PyTorch already has CUDA, and a PyTorch already in the worker's Python is kept as it is, such as one installed with a Tenstorrent card's PyTorch/XLA support. A plain torch keeps working as before.

A block has at most 60 settings.

CUDA with Numba. A block that uses Numba prints a CUDA check in its log before it starts: the Numba, numba-cuda, NumPy and Python it uses, the GPUs the task was given (CUDA_VISIBLE_DEVICES), whether the NVIDIA driver answers, and whether kernels compile. When the driver does not answer, the line says why. Check that nvidia-smi works on the computer; on Windows, if the worker runs as a service and the driver still does not answer, set the service to log on as your own account (services.msc → EsaSim Worker → Log On) and restart it. With numba-cuda, NumPy is kept below 2.5, the first version numba-cuda 0.30 does not work with.

Settings

Settings appear in the block's inspector and reach the code as ctx.params.

TypeShown as
text, long text, promptA text field (prompt: a larger box for instructions to a model). A text setting can instead pick one of the files wired to an input: with "files_from": {"port": "policy", "pattern": "*.pt"} the inspector lists the files of the datasets wired to that input that match, newest first, and an empty choice for the newest at each run. Add "by": "robot", "match": {"Unitree H1": "humanoid_h1_walk_*"} to narrow the list by another setting's choice.
numberA number, with an optional minimum and maximum the editor checks
choiceA list of options. When you rename an option, list the old names in "renamed": {"old option": "new option"}: flows saved with the old name keep working and show the new one.
yes / noA switch
tableRows of values with the columns you name (text, number, choice or colour), each row a card with Add and remove; for example the objects of the 3D scene block. Write the columns as name, speed:number, colour:color, kind:select(a|b). The code gets a list of rows: [{"name": "drone", "speed": 3.5}, …], at most 50 by default.
secretAn API key, pasted once in the block. It is stored encrypted and reaches the code only as an environment variable, ESASIM_SECRET_<KEY>, whose name is in ctx.params["<key>_env"]. With "profile_key": "huggingface", a block left without a key uses the Hugging Face token in the profile of whoever runs the flow (then ctx.params["<key>_from"] is "profile"). Worker blocks only.

Where a worker block runs

"hardware" lists what the block can run on: cpu (any worker), nvidia, amd, intel, ascend, tenstorrent, apple. A run sends the block only to a worker that has one of them, and waits while none is online. When a setting chooses the accelerator, "hardware_by" narrows the list by its option:

"hardware": ["nvidia", "cpu"],
"hardware_by": {"param": "accelerator", "map": {"auto": ["nvidia"], "NVIDIA GPU": ["nvidia"], "CPU": ["cpu"]}}

Here the block goes to an NVIDIA worker unless the person running it picks CPU; options the map does not name keep the whole hardware list.

Worker blocks: def run(ctx)

def run(ctx):
    frames = ctx.inputs.get("frames", [])          # list of input files
    for i, f in enumerate(frames):
        data = open(f.path, "rb").read()            # or f.read_text(), f.read_json()
        ctx.progress(100 * (i + 1) // len(frames), f"{i + 1} of {len(frames)}")
        ctx.telemetry("size", {"step": i, "value": len(data)})
    ctx.write_text("table", "sizes.csv", "frames\n%d\n" % len(frames))
    return {"frames": len(frames)}                  # becomes the "result" output
In the codeWhat it is
ctx.params["key"]The block's settings
ctx.inputs["port"]The files connected to an input: .path, .name, .size, .read_text(), .read_json(), .from_node; from a parallel block .task; from a dataset .rel (its path in the dataset) and .kind
ctx.output("port", "name.ext")The path to write an output file (folders are made); ctx.write_json(port, name, data) and ctx.write_text(port, name, text) write it for you
ctx.telemetry("channel", data)Live values while the block runs (a number, a dict, a list, up to 60 KB), shown in the console's Live tab and by the views wired to the block; they reach the dashboard within about a second
ctx.picture("channel", image)A live picture (a file path, JPEG/PNG bytes, a PIL image or a numpy array; bgr=True for OpenCV), scaled down and sent as JPEG for the Images view
ctx.messages("channel", wait=0)Messages sent from the dashboard while the block runs (Controls and Camera views, or ctx.send in any view): a list of dicts with channel, data and t, oldest first; wait waits up to that many seconds for one. ctx.message("channel", timeout) returns the next one
ctx.progress(pct, "note")Progress on the grid and the run page
ctx.log(...), print(...)Lines of the run's log, in the editor's console
ctx.task.index, .count, .kind (run or combine) for blocks that run as parallel tasks
ctx.llmWith an input of type Model: ctx.llm.chat(messages, system=...) returns .text and the tokens used
ctx.folderThe task's folder on the worker
ctx.time_left(), ctx.max_minutesSeconds left before the run's time limit (Stop after) stops the block, and the limit in minutes; None without a limit. Long blocks use it to finish in time and keep their outputs

The block runs with the worker's Python 3 (or in Docker, when the worker runs blocks there). Packages your code imports must be installed on the worker; Verify lists the ones outside Python's standard library. A worker runs code from projects only if its owner turned on Run code from projects in its Worker settings.

Views: function render(ctx)

function render(ctx) {
  ctx.el.innerHTML = '<div class="viz-pad"><h3 class="viz-h"></h3><pre class="out"></pre></div>';
  ctx.el.querySelector('.viz-h').textContent = ctx.params.title || 'Rows';
  function draw() { ctx.rows('data').then(function (rows) { ctx.el.querySelector('.out').textContent = rows.length + ' rows'; }); }
  ctx.on('update', draw);
  draw();
}

ctx.el is the element to draw into; ctx.input("port") gives { files, stream }; ctx.rows, ctx.json and ctx.text read tables, JSON and text; ctx.files.text / json / buffer / url(file) read one file; ctx.on("update", fn) runs when new data arrives and ctx.on("resize", fn) when the view changes size; ctx.THREE and ctx.uPlot are three.js and uPlot; ctx.save(name, blob) offers a file the view made (a picture, a video) for download or a dataset; ctx.saveFiles([{ name, file }, { name, blob }], title) hands several at once (files of its inputs, or things it made), which the page saves into a dataset as one new version, in a folder, or downloads as one zip (the Video review view saves the kept videos this way). A large file can travel on the live stream in pieces of telemetry that the view puts back together, as Text to video sends whole videos while it runs; the page sends a view only the live items it does not have yet. ctx.send("channel", data) sends a message to the running blocks of the run (they read it with ctx.messages), and ctx.run.active, ctx.run.id and ctx.run.status tell the run shown (ctx.on("run", fn) when that changes). ctx.peer("topic", data) tells the other views on the same page something; they get it with ctx.on("peer", fn), fn(topic, data) (the 3D view's onboard cameras use it to show themselves in the main view). Views run in a sandboxed frame that cannot reach the dashboard or the network: data comes in through ctx. In the code panel, Preview runs the view with sample data.

Views that start runs

A view can start a run of its flow itself, so the View page works like an app: a prompt box and a Generate button, a form of settings and a Simulate button. Tick Starts runs in the view's definition (from an AI assistant: "permissions": ["run"]), then call ctx.run from a click:

button.addEventListener('click', function () {
  ctx.saveState({ prompt: box.value }, true);           // keep what was typed: the new run shows the view again
  ctx.run({ params: { gen: { prompt: box.value, count: 4 } } })   // node id → { setting key: value }
    .then(function (r) { status.textContent = 'Run #' + r.job + ' started'; },
          function (err) { status.textContent = err.message; });
});
  • Settings for this run only. params gives values for settings of the flow's worker blocks: text, numbers (within their range), true or false, and the options of a choice. They go to this run and are written in its log; the flow keeps its own. Secrets, datasets, folders and tables cannot be set this way.
  • Only on a click. ctx.run starts a run only during a click or key press of the person looking at the view, who needs the runner role in the project. One run of the flow at a time: while one goes, it answers with the reason.
  • Where it runs. Like the last run of the flow: its placement or worker, and its Stop after. The first run is placed automatically.
  • What comes back. The view then shows the new run: its result files and live data arrive as in any view, and ctx.run.status follows it to succeeded or failed.

The built-in Prompt view does all this for generative flows: wire the output of Text to image, Text to video or any block with a Prompt setting into it, and it shows a prompt box, more fields if you want them (number count 1 20 4 Images), a Generate button, the last prompts, and the pictures, videos or text the run makes.

The header of the code

Each time a block is saved, EsaSim writes a comment at the top of its code from the definition: the block's name, id and version, its workspace and author, every port with its type and formats, every setting with its default, and how to reach them in the code. You never edit it: change the definition and save, and the header follows. Built-in blocks carry the same header, with EsaSim as publisher.

# ── EsaSim block ──────────────────────────────────────
# Frame counter (ws-frame-counter) · version 3 · machine block, Python: def run(ctx)
# Counts frames and writes a CSV.
# Workspace: Acme Robotics · Author: Maya Chen
# Inputs   ctx.inputs["id"] is a list of files: .path .name .size .read_text() .read_json()
#   frames      Files (png, jpg)          Frames · several
# Outputs  ctx.output("id", "name.ext") is the path to write
#   table       Table (csv)               Counts
# Settings ctx.params["key"]
#   min_size    number    Minimum size = 10
# ── end of EsaSim block header ─────────────────────────

Versions

Each save is a new version, kept with who saved it and when; the block page lists them, each one downloadable. Flows always use the latest version of a workspace block. A block's code can also be changed in one flow only (Edit code in the block's settings): that change stays in that flow.

Share a block with every EsaSim workspace

On a block's page, workspace admins can send its current version to the EsaSim library with a note for the reviewers: what the block is for, what it needs on the worker, and how it was tested. The EsaSim team reads the exact code and definition sent. Once approved, every workspace finds the block in its menu, credited to your workspace; a later version is published only after its own review. Secrets never travel: secret settings are empty in the library. You get an email with the decision.

Keep a copy on GitHub

On a block's page, link your GitHub account (or a personal access token: which permissions to give it), pick a repository, a branch and a folder, and Save to GitHub: block.json, the code and a README go in as one commit in your name. Tick Save every new version there too to commit each new version as it is saved. The link is yours, not the workspace's; tokens are stored encrypted and never shown again.

Under Profile > GitHub you link the account once and choose a default repository, branch and folder; there, Save my blocks by themselves commits every block you save, and you choose where projects go (a repository for each, or folders of one). A project brings the workspace blocks its blueprints use with it, without keys. Save everything now puts all blocks and projects of the workspace in one commit. Projects themselves live on GitHub and are saved there by themselves after every change: see Projects. The repository is laid out as <folder>/<workspace>/blocks/<block>/ and <folder>/<workspace>/projects/<project>/ (with project.json, a README and blueprints/*.json).

With A repository for each project (the default), each project gets a private repository of its own the first time it is saved, holding project.json, a README, blueprints/*.json and blocks/<block>/ for the workspace blocks its blueprints use, and nothing else. Saving opens a window that lists each step as it happens.

Bring projects back: GitHub and zip

A project's Settings has Update from GitHub: the blueprints in the repository replace the drafts of the project's blueprints (matched by blueprint, then by name; published versions are kept), new blueprints are added, and blueprints that are only in EsaSim stay. The workspace blocks the blueprints use come too: a block that is the same here is left alone, one that differs gets a new version (the window says which other projects use it), one that is missing is made. Keys pasted in blocks never travel: the blocks keep the keys they have here.

Export > Download as zip gives the same files as a zip, and Update from a zip takes them back. To start a project from either, open Projects > New project > Import a project and give a repository (and, when it holds several projects, the folder) or the zip.

Let an AI assistant build it

An assistant connected to EsaSim (see AI assistants) can read the block library, create and change blocks with save_block, wire them into flows and run them, with your access and within your role.

Limits

  • Up to 300 blocks per workspace; code up to 300 KB; 12 inputs, 12 outputs and 30 settings per block.
  • A block cannot change from a worker block to a view: make a new one.
  • Deleting a block removes it from the menu; flows that use it show a problem until it is replaced. Past runs are kept.