Storage
Sizing, layout and scaling the ledger in production.
An index lives under .ekos/ beside the sources, or, for a project in the EKOS home, under <home>/data/<tenant>/<project>/state/ next to its ekos.toml. Rough guidance from real runs:
| Workspace | Objects | Ledger entries |
|---|---|---|
| EKOS's own repository (multi-language) | ~16,700 | ~204,000 |
Ledger size grows with history because it is append-only; identical objects are skipped on re-commit.
Choosing a layout
- New workspaces: the fact engine (default).
- Existing SQLite workspaces: keep them; migrate deliberately with
ekos ledger migrate --v3. - Very large or long-lived ledgers: partition by dimension and time bucket (
[storage.partition]). - Multiple machines: distributed mode — a coordinator, compile workers (lease/heartbeat protocol), query workers, and a gateway that merges per-shard statistics and fuses rankings. The object-store read path was verified against MinIO and a 95-partition workspace.
Disk
Keep the state directory on local SSD storage: the fact engine memory-maps segments, and a commit's synchronous writes wait on the disk's latency. Measured in the cloud runs (devlog_315/316):
| Disk | Commit cost |
|---|---|
| Azure premium managed disk | a 142k-object commit took 48 minutes, even at P20 — a 32 GiB premium disk is only a P4 (120 IOPS); keep a VM's home_size_gb at 128 or more |
| Azure Files NFS (Container Apps) | about 90–105 ms per object: 142k objects in about 3½ hours, 2,983 objects in about 5 minutes |
Most of those 142k objects came from data scripts; excluding them by name in [observe] ignore-patterns gave 2,983 objects with the same tables and relationships. Clear .ekos/artifacts/ with ekos clean if only the artifact cache is large.