Open Semantic Data Layer for Agentic Apps

Most enterprises considering a semantic layer for AI agents reach for the same first instinct: stand up a graph database, load a copy of the important tables into it, and let agents query it directly. That instinct is usually wrong. It adds a fourth or fifth data store to an estate that already has Databricks, Snowflake, and a handful of operational RDBMSs, each needing its own pipeline and drift monitoring.

A semantic layer does not have to mean a graph database. You can expose your existing warehouses and lakehouses as a virtual knowledge graph, queryable in standard SPARQL, without moving a single row. This post walks through that architecture: what each layer does, why the mapping stays thin if you do the sequencing right, and how to stand up a local sandbox to validate the pattern before you commit an enterprise budget to it.

Why skip the native graph database

A native graph database like Neo4j is the right call when the graph itself is the primary workload: recommendation traversal, fraud ring detection, or anything where you are writing and re-querying relationships at high frequency. It is the wrong call when the actual goal is giving AI agents and BI tools a consistent, semantically labeled view over data that already lives, and is already governed, in your warehouses.

Loading a copy into a graph store buys you graph query ergonomics at the cost of a new source of truth. Every downstream schema change, every access control update, and every data quality fix now has to be replicated into a system that was never your system of record. For most enterprise semantic layer use cases, that tradeoff is not worth it.

The alternative is virtualization. A translation engine sits between the query language agents want to speak (SPARQL, the W3C graph query standard) and the SQL your data already speaks, rewriting one into the other at query time. No data moves. No second copy exists to drift out of sync.

The four layers of a virtual semantic layer

1. Consumer layer

AI agents built on frameworks like LangGraph, BI tools such as Tableau, and graph retrieval augmented generation applications all speak the same protocol into the layer below: SPARQL over HTTP. This is the point of the architecture. One query interface serves every consumer, instead of each team building a bespoke connector to each warehouse.

2. Semantic middleware

This is the layer that replaces the graph database. Ontop, an open source virtual knowledge graph engine, runs as a stateless, horizontally scalable service. It holds two artifacts: an ontology (the OWL or RDFS vocabulary your organization uses to describe entities like legal entity or client), and a mapping (R2RML or RML rules tying that vocabulary to specific tables and columns). When a SPARQL query arrives, Ontop rewrites it against the mapping and compiles a SQL statement, then reshapes the relational result back into graph triples on the way out.

Because the service is stateless, it scales the way any API tier scales: add containers behind a load balancer. There is no data partition to rebalance and no replication lag to manage.

3. Federated compute layer

The compiled SQL has to run somewhere. Trino handles cross-source federation, joining data that physically lives in Databricks, Snowflake, and a Postgres instance in a single query plan. DuckDB serves the lower-latency case, reading Parquet partitions directly out of Iceberg or Delta Lake tables for agent memory or local microservice patterns where round-tripping through a distributed engine adds latency you cannot afford.

4. Enterprise storage and catalogs

The data never leaves its original platform. Databricks and Snowflake expose curated Gold layer views, registered in an open catalog such as Unity Catalog or Apache Polaris so the query engine can resolve schema without vendor specific plumbing. Operational RDBMSs stay exactly where they are.

What makes the layer semantic rather than just federated

Trino alone gives you federated SQL. What turns that into a semantic layer, something an agent can reason over rather than just query, is a stack of W3C standards sitting on top of the raw triples. Each layer adds a capability the one below it does not have, and each is a separate artifact you author and version, not a single monolithic config file.

Layer	Standard	Adds	Where it lives in this architecture
Model	RDF	The foundational data model: every fact is a subject, predicate, object triple, forming a directed labeled graph	What Ontop emits from every mapped row
Schema	RDFS	Basic vocabulary: classes, properties, domains, ranges, hierarchies	The minimum needed to say `LegalEntity` is a type and `hasLegalName` applies to it
Query	SPARQL	The standard query language for RDF: relationship traversal, pattern matching, federation across graphs	What the consumer layer sends and the SPARQL gateway receives
Reason	OWL	Richer ontologies: class axioms, property characteristics, and inference of facts that were never explicitly stated	The ontology file (`.owl` or `.ttl`) mounted into the Ontop cluster
Validate	SHACL	Shapes and structural constraints that a graph has to satisfy before anything downstream trusts it	The data quality firewall described above, run before generative AI tools ever see the graph

The financial ontology and R2RML mapping in the sandbox above only exercise the bottom three rows: RDF as the output shape, RDFS style class and property declarations, and SPARQL as the query interface. That is enough to prove the pattern, but it is intentionally minimal. Two additions turn it from a working demo into something an agent can use for more than lookups.

OWL is what lets the ontology carry actual business meaning instead of just labels. Declaring LegalEntity and Client as disjoint classes, or that hasParentCompany is transitive, lets a reasoner infer relationships nobody explicitly mapped: which entities roll up to which ultimate parent, without a query author having to write that traversal by hand. For an AI agent, that inference layer is the difference between a graph it can query and a graph it can reason over.

SHACL is what keeps that trust intact as the mapping surface grows. A shape that requires every LegalEntity to have exactly one hasLegalName and a jurisdiction drawn from a fixed code list catches a mapping that compiles cleanly but produces an incomplete or malformed entity, before an agent builds an answer on top of it. Run SHACL validation in the same CI pipeline that validates the ontology, not as a manual step someone remembers to do before a release.

Why the mapping step is the one to get right

The architecture above only stays maintainable if you resist one temptation: mapping directly against raw operational tables. A thousand denormalized columns across a dozen source systems will produce a mapping file that nobody can safely change, and every schema migration in any source system becomes a semantic layer incident.

The fix is a hub and spoke model you likely already run for BI. Curate Gold layer views with the data engineering tooling you already have, dbt or Spark, so a view like v_legal_entities is the single denormalized, business meaningful surface for that concept. Map the ontology to that one view. Every underlying join, every legacy column name, every historical schema decision stays hidden behind it. When a source table changes, you fix the view, not the mapping.

Three practices keep this scalable as the number of mapped concepts grows.

Automated schema bootstrapping. Ontop and Apache Jena can scan primary and foreign key structure and generate a baseline direct graph mapping in minutes. Start from that, then refine, rather than hand authoring every triple pattern from a blank file.
Ontology alignment as a starting draft, not a final answer. Vectorizing catalog descriptions against an industry ontology such as FIBO for financial data, and using that similarity to draft candidate mappings, gives human data stewards a starting point to audit rather than a blank page to fill. Treat the output as a draft. Verify entity and property mappings against your own governance definitions before anything reaches production, since an unreviewed automated mapping is exactly how a wrong label ends up in front of an agent that has no way to know it is wrong.
Data quality firewalls. SHACL, the Shapes Constraint Language, defines structural rules a virtual graph has to satisfy before it reaches a generative AI consumer. This is where you catch a mapping that technically compiles but produces an entity missing a required property.

Treat semantic definitions as code

Ontologies, mappings, and SHACL shapes belong in Git, validated in CI, and deployed the same way your application code is. A typical pipeline runs Apache Jena headless to validate ontology consistency and mapping syntax on every pull request, then packages the validated ontology and mapping files into the containers that mount into your Ontop cluster on merge.

This matters more than it sounds like it should. A semantic layer that lives outside your normal change management process drifts silently: someone edits a mapping file directly in production to unblock a demo, and six months later nobody can explain why one entity type behaves differently from the rest. GitOps discipline here is not process overhead, it is the only thing that keeps a shared semantic model shared.

A local sandbox to validate the pattern

Before proposing this to an architecture review board, run it. The following docker-compose stack stands up a Postgres instance representing a cleaned Gold layer table, an Ontop container, and exposes a working SPARQL endpoint over it in a few minutes.

version: '3.8'
services:
  relational-db:
    image: postgres:15-alpine
    container_name: relational-db
    environment:
      POSTGRES_DB: enterprise_gold
      POSTGRES_USER: data_architect
      POSTGRES_PASSWORD: SecurePassword123
    ports:
      - "5432:5432"
    volumes:
      - ./input_schema.sql:/docker-entrypoint-initdb.d/input_schema.sql

  ontop-middleware:
    image: ontop/ontop:5.1.0
    container_name: ontop-middleware
    ports:
      - "8080:8080"
    environment:
      - ONTOP_ONTOLOGY_FILE=/opt/ontop/financial_ontology.ttl
      - ONTOP_MAPPING_FILE=/opt/ontop/database_mapping.r2rml
      - ONTOP_DB_URL=jdbc:postgresql://relational-db:5432/enterprise_gold
      - ONTOP_DB_USER=data_architect
      - ONTOP_DB_PASSWORD=SecurePassword123
      - ONTOP_DB_DRIVER=org.postgresql.Driver
      - ONTOP_PORT=8080
      - ONTOP_DEV_MODE=true
    volumes:
      - ./financial_ontology.ttl:/opt/ontop/financial_ontology.ttl
      - ./database_mapping.r2rml:/opt/ontop/database_mapping.r2rml
    depends_on:
      - relational-db

Seed a single Gold layer table:

-- Represents a cleaned customer table inside a Gold data warehouse layer
CREATE TABLE gold_corporate_clients (
    client_id VARCHAR(50) PRIMARY KEY,
    legal_name VARCHAR(255) NOT NULL,
    jurisdiction VARCHAR(10) NOT NULL,
    tax_identifier VARCHAR(50)
);

INSERT INTO gold_corporate_clients (client_id, legal_name, jurisdiction, tax_identifier) VALUES
('C-9081', 'Apex Global Holdings Ltd', 'US', 'TAX-112233'),
('C-4412', 'Vertex Asset Management', 'UK', 'TAX-445566');

Define a minimal ontology, modeled on the structure FIBO uses for legal entities:

@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix owl: <http://www.w3.org/2002/07/owl#> .
@prefix fibo-fnd-org-formal: <https://edmcouncil.org/ontology/FND/Organizations/FormalOrganizations/> .
@prefix fibo-fnd-rel-rel: <https://edmcouncil.org/ontology/FND/Relations/Relations/> .

fibo-fnd-org-formal:LegalEntity a owl:Class ;
    rdfs:label "Legal Entity" ;
    rdfs:comment "An organization or individual that can enter into contracts and be held legally accountable." .

fibo-fnd-rel-rel:hasLegalName a owl:DatatypeProperty ;
    rdfs:label "has legal name" ;
    rdfs:domain fibo-fnd-org-formal:LegalEntity ;
    rdfs:range <http://www.w3.org/2001/XMLSchema#string> .

And the R2RML mapping tying the table to that ontology:

@prefix rr: <http://www.w3.org/ns/r2rml#> .
@prefix fibo-fnd-org-formal: <https://edmcouncil.org/ontology/FND/Organizations/FormalOrganizations/> .
@prefix fibo-fnd-rel-rel: <https://edmcouncil.org/ontology/FND/Relations/Relations/> .

<#ClientMappingRule>
    rr:logicalTable [ rr:tableName "gold_corporate_clients" ] ;

    rr:subjectMap [
        rr:template "http://enterprise.com/client/{client_id}" ;
        rr:class fibo-fnd-org-formal:LegalEntity ;
    ] ;

    rr:predicateObjectMap [
        rr:predicate fibo-fnd-rel-rel:hasLegalName ;
        rr:objectMap [ rr:column "legal_name" ] ;
    ] .

Bring the stack up with docker-compose up -d, then query the virtual graph:

curl -X POST http://localhost:8080/sparql \
  -H "Accept: application/sparql-results+json" \
  -H "Content-Type: application/sparql-query" \
  -d '
  PREFIX fibo-fnd-org-formal: <https://edmcouncil.org/ontology/FND/Organizations/FormalOrganizations/>
  PREFIX fibo-fnd-rel-rel: <https://edmcouncil.org/ontology/FND/Relations/Relations/>

  SELECT ?client ?legalName
  WHERE {
      ?client a fibo-fnd-org-formal:LegalEntity ;
              fibo-fnd-rel-rel:hasLegalName ?legalName .
  }'

Check the Ontop container logs and you will see no graph engine ever runs. The SPARQL query is rewritten into ordinary SQL and pushed down to Postgres:

SELECT
  CONCAT('http://enterprise.com/client/', client_id) AS client,
  legal_name AS legalName
FROM gold_corporate_clients;

That is the entire architecture proven end to end on a laptop: a graph query answered by a relational table, with the ontology and mapping as the only new artifacts.

Putting the pattern to work

Open semantic data layer architecture, from AI agents and BI tools through a SPARQL gateway and Ontop to Trino and the underlying Databricks, Snowflake, and Postgres stores

Scaling this from a laptop sandbox to production comes down to the same discipline you already apply to data pipelines: curate before you map, validate before you deploy, and treat the semantic model as versioned code rather than a one time configuration exercise. If your organization is already running dbt or Spark to produce Gold layer views for BI, most of the hard work is done. The semantic layer adds one more consumer of those views rather than a new pipeline to maintain.

Two decisions shape how this plays out in practice. First, what your Gold layer actually looks like today: a warehouse heavy estate on Databricks and Snowflake maps cleanly through Trino, while an estate still leaning on operational RDBMSs for reporting needs the curated view step done first, not skipped. Second, what your primary consumer is: an agent framework like LangGraph benefits from the ontology carrying business meaning an LLM can reason over, while a BI dashboard mostly needs consistent joins across sources it could not previously join at all. Both are legitimate entry points into the same architecture. Pick the one that matches where your immediate pain is, and let the other follow once the pattern is proven.

Decision tree for choosing a semantic data layer architecture: native graph database if traversal is the primary workload, curate Gold views first if the source data is not yet governed, then federate directly with Trino or DuckDB or virtualize with Ontop depending on whether consumers need agent grade reasoning

For related grounding on the data engineering side of AI systems, see the data engineer to AI data engineer learning path, and for how agents consume this kind of layer once it exists, see emerging multi agent orchestrator design patterns and what is context engineering.

Key questions

Q1) Do I need a native graph database like Neo4j to build a semantic layer for AI agents?

No. A virtual knowledge graph engine such as Ontop translates SPARQL queries into SQL at query time and pushes them down to your existing warehouses and RDBMSs. You only need a native graph database when the graph traversal itself is the primary workload, not when the goal is a semantically labeled query interface over data that already lives elsewhere.

Q2) What is the biggest risk in a virtual semantic layer implementation?

Mapping directly against raw, ungoverned tables instead of curated Gold layer views. That approach ties the mapping file to every schema decision in every source system, so a routine migration anywhere in the estate becomes a semantic layer incident. Curate a business meaningful view first, then map the ontology to that view.

Q3) Does the semantic layer move or duplicate my data?

No. The engine rewrites the incoming graph query into SQL and executes it against your existing storage through a federated query engine like Trino or DuckDB. Nothing is copied and nothing new becomes a source of truth.

Building an Open Semantic Data Layer for Agentic Apps