Hugging Face dataset
Reads files of a dataset on the Hugging Face Hub at a fixed revision, all or a repeatable sample, and records the exact commit read.
How it works
Repository: owner/name or the dataset's link, such as imageomics/GZCD. Revision: a branch, a tag or a commit (default main); it is resolved to its commit, so the run's records say exactly which version of the data it read, and reading the same commit again gives the same files. Keep: patterns of the files to read (images/*.jpg); Also: files read every time and given apart (metadata, annotations, the card).
Most files: a random but repeatable sample (Seed), for a demo-sized set. Files are downloaded once into the worker's cache. Public datasets need no account; for a private one, paste a token with read access. Respect the dataset's licence: it is in the manifest.
Outputs
| Output | Gives | What it is |
|---|---|---|
Filesfiles | Files | The files read, in their folders |
Other filesextra | Files | The files of Also (metadata, annotations) |
Manifestmanifest | Table csv, json | manifest.csv (path, size, sha256) and source.json (repository, commit, link, licence) |
Live telemetrytelemetry | Live telemetry | What the block sends while it runs, for views such as Chart or 3D view. |
Settings
| Setting | Default | What it does |
|---|---|---|
Repositoryrepo |
empty | owner/name or https://huggingface.co/datasets/owner/name |
Revisionrevision |
main | A branch, a tag or a commit |
Keepkeep |
*.jpg, *.jpeg, *.png | Patterns of the files to read, separated by commas (images/*.jpg) |
Alsoalso |
empty | Files to read every time, apart: metadata, annotations (final_metadata.json, README.md) |
Most filesmax_files |
0 | A random sample of at most this many files; 0 reads them all From 0 to 1000000 |
Seedseed |
1 | |
Kindrepo_type |
dataset | One of: dataset, model |
Token (private datasets only)token |
pasted in the editor, kept encrypted | Empty: the Hugging Face token in the profile of whoever runs the flow (needed for private or gated datasets only) |
Where it runs
On one of your computers with the EsaSim worker: Any worker.
The worker installs the Python packages it needs when they are missing: huggingface_hub.
In its code, settings are ctx.params["key"], inputs ctx.inputs["id"] and outputs ctx.output("id", "name.ext"); see Custom blocks.