Apache Ossie

Export the compiled schema as an Apache Ossie semantic model, read the Ossie models a team keeps in its repositories, and turn a reviewer's edits to an export into review decisions.

Apache Ossie (incubating) is the new name of the Open Semantic Interchange (OSI). It is a vendor-neutral YAML/JSON format for semantic models, and Snowflake, dbt, ThoughtSpot, Dremio, Power BI and others read and write it. An Ossie model holds:

• datasets: a table or view, its typed fields, primary and unique keys, and its own metrics;
• relationships between datasets;
• model-level metrics;
• ai_context: instructions and synonyms for AI tools.

EKOS works with Ossie in three ways:

Availability. Apache Ossie support is on main (RFC 0245, 2026-11-07). It is not in release v1.3.0; build from source to use it now. EKOS writes spec 0.2.0.dev0, the version the official schema on Ossie's main defines, and reads the released 0.1.1 too. Ossie marks 0.2.0.dev0 as still changing.

Export

ekos export ossie --out shop.ossie.yaml                  # meanings: confirmed only (the default)
ekos export ossie --status all --out shop.ossie.yaml     # every meaning, each marked with its status
ekos export ossie --json --out shop.ossie.json           # JSON instead of YAML (or end --out in .json)
ekos export ossie --database erp --out erp.ossie.yaml    # only tables of one [[databases]] entry
ekos export ossie --source billing                       # only tables of one [[sources]] entry

--name sets the model's name. It defaults to the workspace directory's name.

How EKOS objects map to Ossie

EKOS never invents a metric. Metrics appear in an export only when a team's own Ossie file defines them. A business concept such as open shipments is a filter, not an aggregation, so EKOS describes it in ai_context and never turns it into a COUNT(*). Meanings a person has not reviewed say "unreviewed" in words, so an AI tool reading the model cannot mistake a hypothesis for a definition.

The EKOS extension

Every element carries one custom_extensions entry with vendor_name: EKOS. Its data is a JSON string, as the spec requires, and holds:

• the element's ledger ekos_id;
• its status (observed for schema facts);
• origin, dialect and evidence (path:line);
• for datasets and fields, the review entries they were built from: concepts and enum_meanings, each with its own ekos_id and status.

Other Ossie tools carry the extension through and ignore it. ekos import ossie reads it back.

A dataset from a real export, with one confirmed concept, one confirmed code and one still unreviewed:

- name: shipments
  source: shipments
  primary_key: [id]
  ai_context:
    instructions: 'Business concept "OpenShipments": shipments.state IN (''P'')'
  fields:
  - name: id
    expression: {sql: id}
    datatype: Integer
  - name: state
    expression: {sql: state}
    description: P = pending, S = shipped, D = delivered
    datatype: String
    ai_context:
      instructions: 'Codes: ''P'' = pending (hypothesis, unreviewed); ''S'' = shipped to customer.'
    custom_extensions:
    - vendor_name: EKOS
      data: '{"enum_meanings":[{"ekos_id":"6663…","label":"pending","status":"hypothesis","value":"''P''"},
             {"ekos_id":"8853…","label":"shipped to customer","status":"confirmed","value":"''S''"}]}'
  custom_extensions:
  - vendor_name: EKOS
    data: '{"concepts":[{"ekos_id":"bad1…","name":"OpenShipments","status":"confirmed", …}],
           "ekos_id":"e0a7…","kind":"Table","status":"observed","evidence":["db/shipments.sql:5"]}'

Validity

Built-in checks. Every export is checked twice before it is written:

• Official schema: EKOS's tests validate every export against Ossie's official JSON Schema, a copy of core-spec/ossie-schema.json.
• Structural checks: the export runs the structural checks of Ossie's own validator: unique names, keys and relationship columns that name declared fields, matching arity. The export fails rather than write a model that does not pass.

Public validation suite. scripts/ossie-validate.sh in the EKOS repository checks EKOS with Ossie's own validator, validation/validate.py. That validator checks the JSON Schema, names, references and arity, and parses every expression with sqlglot. The suite runs on tests/fixtures/ossie-tpcds: five TPC-DS tables, plus Ossie's official TPC-DS example model of them, unchanged.

Run it yourself. It clones apache/ossie at the pinned commit and needs uv:

git clone https://github.com/alexeyban/EKOS && cd EKOS
scripts/ossie-validate.sh --out ossie-report     # every export, validator log and summary.md
OSSIE_REF=main scripts/ossie-validate.sh         # against Ossie's latest spec

CI runs the suite in .github/workflows/ossie.yml against two Ossie versions: - the pinned Ossie commit, a required check; - Ossie's main branch, weekly, as an early warning while the spec is still changing.

Each run uploads its report as an artifact. cargo test -p ekos --test ossie_e2e runs the same flow offline against the copied schema.

To check one file of your own:

git clone https://github.com/apache/ossie
uv run --no-project --with pyyaml --with jsonschema --with sqlglot \
  python ossie/validation/validate.py --schema ossie/core-spec/ossie-schema.json -- shop.ossie.yaml
# Validation PASSED: shop.ossie.yaml

Ingest

ekos recover reads every Ossie model it finds under a project's sources:

• every file named *.ossie.yaml, *.ossie.yml or *.ossie.json;
• any other YAML or JSON file that looks like an Ossie model, with an Ossie version and datasets or semantic_model.

Both spec versions are read. A 0.1.1 file holding several models yields every one of them, never only the first. Like every raw input, the files are redacted before anything is stored.

Each file becomes these objects, keyed by file, so two files never merge:

ekos commit then compares each file with the schema:

• Link. Each dataset is linked to the one table or view its source names, by its full name, then by its last part. A name that matches no table, or more than one, links nothing. commit prints how many datasets linked:

Ossie datasets : 1 linked to a table, 0 unresolved, 0 ambiguous

• Check. Where the file and the schema disagree, the disagreement becomes an ossie_vs_schema conflict, with the file and line of each claim:

$ ekos conflicts list --type ossie_vs_schema [open] Orders.columns.Channel — 2 source(s) disagree: "absent" vs "present" [open] Orders.columns.ShipCountry.datatype — 2 source(s) disagree: "VARCHAR(15)" vs "Integer"

• A missing column: a field names a column the table does not have. This is checked only against tables whose column list is complete, from DDL or a live catalog, never a dbt model's partial documentation.

• A different type: a declared datatype belongs to another family than the column's type. The families are numbers, date-times, dates, times, strings and booleans; Integer against NUMERIC is not a dispute. A boolean against a small integer is not one either, since MySQL and Oracle store flags that way.

• Search. Synonyms become searchable, so ekos query find "purchases" finds the Orders dataset of a model that lists purchases as a synonym.

• Export. The next export of the table carries the file's synonyms, instructions and field descriptions. It also carries the dataset's metrics, but only when every field of the Ossie dataset is a table column under its own name; otherwise a metric could refer to a field EKOS's dataset does not have.

Two safeguards keep this clean:

• Exports are skipped. A file that ekos export ossie wrote is never read back as a source, so EKOS's own output cannot return as if a person had written it.
• Deleted files drop out. recover records which Ossie files it found, in ossie/current.json in the state directory. Linking, conflicts, export and the MCP tool ignore the objects of any file that is gone. The append-only ledger keeps those objects, but nothing reads them.

Import: review inside the file

ekos export ossie --status all --out review.ossie.yaml   # send this to the reviewer
ekos import ossie review.ossie.yaml --dry-run            # the decisions the edits stand for
ekos import ossie review.ossie.yaml --as ann             # record them, all or nothing

The reviewer edits the EKOS extension entries, which are JSON inside the YAML:

Entries are matched by ekos_id, never by name, so renaming a dataset in the file changes nothing. EKOS rebuilds the export the file came from (its status filter and any --database/--source selection are recorded in the model's EKOS extension) and compares the two.

Some edits are only reported. A changed field description, a new type, a new field or metric, or a new dataset is a fact about the schema, not a decision, so to change one, change the source or keep an Ossie model in the sources. An unknown ekos_id or a status other than confirmed or rejected is an error, and if there is any error nothing is written.

ekos import ossie is human-only, like ekos import linkml and ekos semantics confirm. No MCP tool can reach it, and a test that scans the MCP server's source enforces that.

$ ekos import ossie review.ossie.yaml --dry-run --as ann
reject   datasets.shipments.fields.state.'D'  status: hypothesis → rejected
edit     datasets.shipments.fields.state.'S'  label: shipped → shipped to customer
confirm  datasets.shipments.concepts.OpenShipments  status: hypothesis → confirmed
warning: datasets.shipments: edits to the observed schema (fields, types, keys, descriptions, metrics) are not imported — …
3 decision(s) planned, 0 applied (dry run).

For agents: ekos_ossie_export

The MCP server has a read-only tool that returns the model directly, so an agent can see a table's keys, types, relationships and reviewed meanings before it writes SQL or a semantic-layer definition:

The tool reads only through the store, so it also works for a reader whose read scope shows part of a project: that reader gets a model of the part it may see.

In the web console

The Semantics tab has an Apache Ossie view: - Browse: pick a dataset to see its fields, types, meanings, metrics and relationships. - Download: save the model as .ossie.yaml or .json. - Import: paste or load an edited export, then click Validate to run a dry run and see the decisions it stands for. Apply records them as the signed-in reviewer, and needs the review role.

What EKOS does with your model

An imported model is a team's stated knowledge, and five parts of EKOS use it. A single resolver rewrites the model over the compiled tables: - metrics in terms of table columns (a field becomes its expression, dataset.field becomes the table's column); - joins as column pairs; - keys, synonyms and time axes.

Anything it cannot resolve is listed with the reason: an unknown field, a metric built on another metric, or a dataset that is not linked to a table. The model never overrides the schema; where the two disagree, the conflict stands. These consumers are on main (RFC 0246).

Measured on text-to-SQL. The setup was 12 questions about the TPC-DS fixture, 3 runs each, on a small local model (qwen2.5-coder:3b). With the official Ossie model in the sources, ekos sql generate answered 27 of 36 correctly; without it, 24 of 36.

• Gains: "customer lifetime value" and "store productivity" went from 0/3 to 3/3, because the agent used the model's two-table metric definitions.
• Loss: one question went from 3/3 to 0/3, where the agent dropped a join.

This shows that the definitions reach the model and get used; 12 questions on one small model is not a benchmark. The harness is in evals/text2sql/ossie-tpcds/.

Use cases

1. Start a semantic layer from the schema you already have

A team moving to a semantic layer (Snowflake semantic views, dbt MetricFlow, a BI tool) usually starts with an empty file and retypes table names, keys and joins by hand. EKOS writes that part from what it compiled: every table, typed field, key and foreign key, with the evidence for each.

ekos build && ekos recover && ekos resolve && ekos compile && ekos commit
ekos export ossie --out warehouse.ossie.yaml

Then convert the file for the target tool. The official converters live in apache/ossie-converters; EKOS does not run them. For dbt, EKOS writes the semantic layer itself (ekos export dbt-semantic, above). The team adds the metrics; EKOS does not guess them.

2. Document a legacy database for a new BI platform

For an Oracle, PostgreSQL or ClickHouse database read live, export only that database:

ekos database add erp --kind oracle --connect db:1521/ERP --user reader --password-env ERP_PW --owner SALES   # once
ekos export ossie --database erp --status confirmed --out erp.ossie.yaml

Column comments become descriptions, constraints become keys, and the code meanings a person confirmed ('C' = closed by finance) travel with the fields. The BI team receives a documented model instead of a column list.

3. Keep a hand-written Ossie model honest

Teams that keep their Ossie model in the repository next to the DDL can let EKOS check that the two still agree. Every commit flags a field whose column was dropped or renamed, and a type the schema contradicts. A CI job can fail on any open disagreement:

ekos build && ekos recover && ekos resolve && ekos compile && ekos commit
open=$(ekos conflicts list --type ossie_vs_schema --status open --json | jq length)
if [ "$open" -gt 0 ]; then
  ekos conflicts list --type ossie_vs_schema --status open
  exit 1
fi

ekos conflicts show <name> gives the file and line of both claims. A person settles each one with ekos conflicts resolve, or by fixing the file.

4. Let a domain expert review meanings without learning EKOS

EKOS recovers code meanings and business concepts from filters, CHECK constraints, lookup rows and comments (business semantics). Each one is a hypothesis until a person reviews it. The expert may know the business but not EKOS:

ekos export ossie --status all --out for-review.ossie.yaml    # mail or share this file
# … the expert sets "status": "confirmed" / "rejected", fixes labels, adds notes …
ekos import ossie for-review.ossie.yaml --dry-run             # check what the edits mean
ekos import ossie for-review.ossie.yaml --as "maria.lopez"    # record them as her decisions

The decisions are recorded under the reviewer's name, so a later ekos export ossie (default --status confirmed) contains only what she confirmed. The console's Apache Ossie view does the same validate-then-apply in the browser.

5. Ground an agent in the business vocabulary

An agent asked about "purchases by region" needs to know that purchases means the Orders table and which column holds the region. With the team's Ossie model in the sources, the synonyms are searchable and the model is one tool call away:

• ekos_search with purchases finds the Orders dataset through its synonym.
• ekos_ossie_export with match: "orders" returns its fields, keys, relationships and meanings, each meaning with its review status.
• ekos_sql_check then checks the SQL the agent writes against the compiled schema.

6. Publish only reviewed definitions

Run the export as a release step and publish the file where other tools pick it up:

ekos export ossie --status confirmed --out dist/semantic-model.ossie.yaml

The default --status confirmed keeps unreviewed meanings out, so nothing EKOS only inferred leaves as a fact. Each element's EKOS extension still points back to the ledger id and evidence, so a consumer can trace any line to its source.

Limits

• Spec still moving. 0.2.0.dev0 may change before Ossie releases 0.2.0, so EKOS will update its copy of the schema then.
• Model-level metrics from files are not exported. They refer to the file's dataset names, not EKOS's, so only dataset metrics are carried.
• No ontology or mapping documents. Ossie's ontology/ documents are not read or written yet.
• One table per dataset. A dataset whose source is a query, rather than a table or view, is ingested but not linked.
• No live re-check of metric SQL. A metric's expression is carried as written; EKOS does not run or parse it.