EKOSdocs
Docs / Architecture Learn

Compilation Pipeline

The five stages, what each reads and writes, and how caching works.

The five pipeline stages build, recover, resolve, compile and commit, each with the artifact it writes
The five pipeline stages build, recover, resolve, compile and commit, each with the artifact it writes
Stage Reads Writes Notes
build observe paths observation artifacts; File objects redaction runs here; fingerprint cache skips unchanged trees
recover observation artifacts KIR knowledge artifacts analyzers are CompilerPasses scheduled by dependency; --parallel
resolve KIR identity decisions exact merges; fuzzy → unconfirmed SameAs; conflicts fail unless --force
compile resolved KIR CKM (.ekos/ckm/model.json.zst) derives concentration risks; pass output cached by version + inputs
commit CKM ledger entries, indexes then rollups, data lineage, optional LLM descriptions, embeddings, extension steps

Passes

A CompilerPass declares a name, dependencies, a version string and cache inputs. The PassManager orders passes by dependency and skips a pass whose version and inputs are unchanged — the skipping pass (cached) log line. Bump a pass's version when its behaviour changes, otherwise stale output is reused.

Analyzers

Analyzer Source
sql_analyzer, sql_transform_analyzer DDL; SELECT/VIEW/procedures into the Transformation IR
pentaho_analyzer .ktr / .kjb
rust_analyzer, python_analyzer, javascript_analyzer, elixir_analyzer real AST parsing, symbols and Calls
dbt_analyzer dbt models and schema.yml / sources.yml
git_analyzer, github_analyzer, confluence_analyzer history, issues, wiki
local_docs_analyzer, document_semantics_analyzer documents; optional LLM semantics
crate_topology_analyzer, cicd_analyzer, dependency_analyzer, crypto_analyzer structure and dependencies

Post-commit steps

These run after the ledger is written and are not compiler passes: subsystem rollups, data-lineage linking, [llm-description] (opt-in), [embeddings] vectors (opt-in), then extension steps.

Determinism

Same inputs, same outputs. Ids are derived, not random, where identity matters. Extensions that change observer output bump logic_version so the build cache invalidates.