Curate and split
Removes the images the checks flagged (or a person rejected), balances the classes and splits train and validation, with a before/after report.
How it works
Connect the images and the tables of the checks (quality, duplicates, labels: any table with path and flag). Every flagged image is removed, unless Remove names only some flags (such as duplicate, broken, blurry). A Review table (path, decision accept or reject, and optionally a new label) overrides the checks: accepted images stay, rejected ones go, relabelled ones move.
The rest is split into train and validation by class (Validation %), at most Most per class each (0: all), in a repeatable way (Seed), as train/<class>/… and val/<class>/… with manifest.csv. The report gives the images in, removed by reason, kept, and the classes before and after (curation.md, curation.json and CSV tables for charts); removed.csv lists every image removed and why.
Licence and credit files travel with the images.
Inputs
| Input | Takes | What it is |
|---|---|---|
Imagesimages | Files jpg, jpeg, png, webp, bmp, tif | Images, in folders (the folder is the label) Takes several connections. |
Flagsflags | Table csv | Tables with path and flag (quality, duplicates, labels) Optional. Takes several connections. |
Reviewreview | Table csv | path, decision (accept or reject), optionally label Optional. Takes several connections. |
Outputs
| Output | Gives | What it is |
|---|---|---|
Curatedfiles | Files | train/<class>/… and val/<class>/… with manifest.csv |
Removedremoved | Table csv | removed.csv: path, label, flags, reasons |
Reportreport | Table md, json, csv | curation.md, curation.json, curation_by_reason.csv, curation_by_class.csv |
Live telemetrytelemetry | Live telemetry | What the block sends while it runs, for views such as Chart or 3D view. |
Settings
| Setting | Default | What it does |
|---|---|---|
Removeremove |
empty | Flags that remove an image, separated by commas (duplicate, broken, blurry, too_dark, label, outlier); empty: every flag |
Validation (%)val_pct |
20 | From 0 to 90 |
Most per classmax_per_class |
0 | 0: all From 0 |
Layoutlayout |
split/class | One of: split/class, as it was |
Seedseed |
1 |
Where it runs
On one of your computers with the EsaSim worker: Any worker.
In its code, settings are ctx.params["key"], inputs ctx.inputs["id"] and outputs ctx.output("id", "name.ext"); see Custom blocks.