System Overview
The components of EKOS and how knowledge flows from sources to agents.
EKOS is a Rust (2024 edition) Cargo workspace whose root is ekos/. It is a compiler, not a database: it observes, compiles and stores; a separate read-only runtime serves.
Components
| Component | Crate(s) | Role |
|---|---|---|
| Observation | observation-sdk, plugins/* |
The Observer contract and the connectors; produce content-addressable artifacts |
| Artifacts | artifact |
Artifact types and the store (loose JSON or packed segments) |
| Compiler core | compiler-core, scheduler, compiler-sdk |
Compiler, PassManager, scheduling, diagnostics, EkosConfig |
| Live catalogs | ora-live, pg-live, plugins/oracle, plugins/postgres, plugins/clickhouse |
Read Oracle, PostgreSQL and ClickHouse catalogs in read-only sessions, with their own redaction |
| Recovery | recovery |
One analyzer per source kind plus LLM provider glue |
| Procedural IR | plpgsql, plsql |
PL/pgSQL and PL/SQL parsers into one procedural IR |
| IR | kir |
Typed intermediate representation; the registry of custom kinds |
| Identity | identity |
Same-source merging and cross-system candidate scoring |
| Semantic compiler | semantic |
KIR → CKM; the shared Transformation IR; concentration risks; lineage; conflicts; write behaviour; review subjects and the workflow gate; the cross-reference integrity check |
| Ledger | ledger, segment-backend, cluster, distributed |
Append-only store, indexes, partitioning, distribution |
| Runtime | runtime |
Read-only state reconstruction, retrieval, ekos ask grounding |
| Query language | ekl |
Parser and interpreter |
| Output | docs-gen, dbt-gen |
Deterministic docs (including estate documentation) and dbt export |
| Interface | cli |
Every subcommand and the MCP server; the extension seam |
| Migration | migrate, migrate-* |
EKOS Migrate (opt-in): PostgreSQL → ClickHouse with measured validation and human-only approval |
| Session memory | session |
Agent notes written to an inbox file, compiled into the ledger by ekos session commit; confirming one is human-only (opt-in) |
| Evaluation | evals |
Grades whole ekos ask answers |
| Shared | common |
Hashing, compression, redaction, PII rules, run-log events |
| Web console | web/api (Python, FastAPI), web/ui (React) |
Projects, roles, review, runs, chat and the MCP gateway for agents; reads through each project's MCP server and makes changes by running ekos commands |
Auxiliary crates outside the pipeline: simulation (World Engine), marketing. Extensions such as EKOS Lakehouse live in their own repositories and plug in through the extension seam.
Invariants
- The ledger is append-only.
- Every conclusion is traceable to evidence.
- The runtime is read-only; agents use it, never the raw systems.
- Compiler passes are deterministic and side-effect-free.
- Artifacts are content-addressable.
- Secrets and PII are never observed or stored.
- Sessions against a source database are read-only, and source trees are never written.
- Review decisions, sign-offs and approvals are human-only: no MCP code can reach them.
Deployment shape
All state lives in one directory, the EKOS home: every project's config and compiled data, the console's database and run logs. The same pieces run in three ways:
| Shape | What runs |
|---|---|
| One machine | The ekos binary and, optionally, the console it starts (ekos setup, ekos console) |
| Container | The production image: the binary, the console API and UI on one port, the home on one volume, sources mounted read-only, sign-in through any OIDC provider |
| Cloud | Terraform modules for Azure, Google Cloud and AWS, each as a VM, a serverless container or Kubernetes (with a Helm chart); secrets in the cloud's secret store, sources synced from git, nightly backups of the home to object storage |
The console holds an instance lease on the home, so a shared home has one writer, and on first start over an empty home it restores the newest backup. Of the cloud shapes, the Azure VM and Azure Container Apps have been applied to a real subscription; the others are checked by terraform validate, mocked plans and helm lint. See Deployment.