EKOSdocs
Docs / Architecture Learn

System Overview

The components of EKOS and how knowledge flows from sources to agents.

EKOS system overview: sources, compiler, ledger, read-only runtime, MCP and CLI interfaces, agents
EKOS system overview: sources, compiler, ledger, read-only runtime, MCP and CLI interfaces, agents

EKOS is a Rust (2024 edition) Cargo workspace whose root is ekos/. It is a compiler, not a database: it observes, compiles and stores; a separate read-only runtime serves.

Components

Component Crate(s) Role
Observation observation-sdk, plugins/* The Observer contract and the connectors; produce content-addressable artifacts
Artifacts artifact Artifact types and the store (loose JSON or packed segments)
Compiler core compiler-core, scheduler, compiler-sdk Compiler, PassManager, scheduling, diagnostics, EkosConfig
Live catalogs ora-live, pg-live, plugins/oracle, plugins/postgres, plugins/clickhouse Read Oracle, PostgreSQL and ClickHouse catalogs in read-only sessions, with their own redaction
Recovery recovery One analyzer per source kind plus LLM provider glue
Procedural IR plpgsql, plsql PL/pgSQL and PL/SQL parsers into one procedural IR
IR kir Typed intermediate representation; the registry of custom kinds
Identity identity Same-source merging and cross-system candidate scoring
Semantic compiler semantic KIR → CKM; the shared Transformation IR; concentration risks; lineage; conflicts; write behaviour; review subjects and the workflow gate; the cross-reference integrity check
Ledger ledger, segment-backend, cluster, distributed Append-only store, indexes, partitioning, distribution
Runtime runtime Read-only state reconstruction, retrieval, ekos ask grounding
Query language ekl Parser and interpreter
Output docs-gen, dbt-gen Deterministic docs (including estate documentation) and dbt export
Interface cli Every subcommand and the MCP server; the extension seam
Migration migrate, migrate-* EKOS Migrate (opt-in): PostgreSQL → ClickHouse with measured validation and human-only approval
Session memory session Agent notes written to an inbox file, compiled into the ledger by ekos session commit; confirming one is human-only (opt-in)
Evaluation evals Grades whole ekos ask answers
Shared common Hashing, compression, redaction, PII rules, run-log events
Web console web/api (Python, FastAPI), web/ui (React) Projects, roles, review, runs, chat and the MCP gateway for agents; reads through each project's MCP server and makes changes by running ekos commands

Auxiliary crates outside the pipeline: simulation (World Engine), marketing. Extensions such as EKOS Lakehouse live in their own repositories and plug in through the extension seam.

Crate layers: common at the bottom, cli at the top
Crate layers: common at the bottom, cli at the top

Invariants

  • The ledger is append-only.
  • Every conclusion is traceable to evidence.
  • The runtime is read-only; agents use it, never the raw systems.
  • Compiler passes are deterministic and side-effect-free.
  • Artifacts are content-addressable.
  • Secrets and PII are never observed or stored.
  • Sessions against a source database are read-only, and source trees are never written.
  • Review decisions, sign-offs and approvals are human-only: no MCP code can reach them.

Deployment shape

All state lives in one directory, the EKOS home: every project's config and compiled data, the console's database and run logs. The same pieces run in three ways:

Shape What runs
One machine The ekos binary and, optionally, the console it starts (ekos setup, ekos console)
Container The production image: the binary, the console API and UI on one port, the home on one volume, sources mounted read-only, sign-in through any OIDC provider
Cloud Terraform modules for Azure, Google Cloud and AWS, each as a VM, a serverless container or Kubernetes (with a Helm chart); secrets in the cloud's secret store, sources synced from git, nightly backups of the home to object storage

The console holds an instance lease on the home, so a shared home has one writer, and on first start over an empty home it restores the newest backup. Of the cloud shapes, the Azure VM and Azure Container Apps have been applied to a real subscription; the others are checked by terraform validate, mocked plans and helm lint. See Deployment.