Enterprise Context Graph Reference Architecture for Agentic Enterprise Search

Enterprise AI Assistants are challenged with providing the appropriate level of context to ensure that the frontier models have the enterprise context to accomplish the assigned objective.

An enterprise graph augments the intelligence layer that not only builds a dynamic understanding of the the problem, but also provided meaningful context based on the relationships between relevant entities (people, content, workflows, conversations, and more).

How it works

The enterprise graph layer holds four things, not two:

• Business ontology — your enterprise’s model of itself. Concepts, processes and policy mapped to systems and data assets. “Active customer” means one governed thing instead of whatever a status code implies.
• Personal graph per employee — their projects, collaborators and work style, so ranking is personalized without duplicating meaning in every agent’s prompt.
• Entitlement graph — who can see what, enforced at retrieval time before the model ever sees the document.
• Decision memory — every agent run leaves a trace. The next run inherits what the last one learned instead of re-deriving it from scratch.

Retrieval, ranking and agent planning all read from this one layer. That is why the same question gets correctly different answers for different personas, and why the tenth agent built against it starts from everything the first nine already learned.

Above the graph sits a two-stage reasoning path, not one model doing everything:

1. A small, reinforcement-learned agentic search model runs first. It executes a short, fixed tool plan over enterprise search, employee search and web search, then hands off to the frontier model with context already assembled.
2. The frontier model is invoked once, already grounded, instead of driving its own multi-turn tool loop. Production deployments see roughly half the latency and a quarter the token spend.

The frontier model plans, decides and executes multi-step work—adapting as it goes—without the employee hand-tuning prompts per task.

Openness is architectural, not optional. Developers and employees choose their preferred models, frameworks and tools. Permissions, security and governance are enforced once, centrally, below that choice—not re-implemented per integration.

Model routing in practice

Frontier Model Cost and Open Weights Popularity is Driving Demand for Model Routing - Arvind Jain, Glean CEO

Automated model routing saves cost and latency without forcing a manual choice on every call. In practice it offers three levels of selection. A mature deployment supports all three rather than picking one:

The key: all three fire independently. An admin restriction does not require every employee to make a choice. Automatic mode does not prevent explicit selection when it matters.

Lighter agents, a smarter shared substrate

The obvious way to give an agent business meaning is to build it in:

• A definition in the system prompt
• A custom tool in its MCP server
• A retrieval pipeline tuned to fetch the right paragraph

Ontology, entitlements, decision memory—is defined once in the enterprise graph and read from there by every agent, tool and search call. Agents shrink to what they’re good at: interpreting intent, planning, executing. The graph carries the weight of understanding the business. That understanding stays in one place instead of forking into every integration that touches it.

What to get right

• Filter by entitlement before ranking, not after generation. Filtering after the model has already seen a document does not reliably keep it out of the answer — enforce access at retrieval time, sourced from the entitlement graph, every time.
• Keep the search model small and the tool plan fixed. The efficiency case for an agentic search model collapses if it is allowed to freelance; a short, bounded plan over a known toolset is what makes the latency and token savings predictable.
• Treat the personal graph as a signal, not a profile. It should shift ranking and relevance, not silently expand what an employee can retrieve — personalization and entitlement are separate mechanisms even though they both read the same graph.
• Route models under policy, not by default. Automatic selection needs an admin-set ceiling on cost and capability behind it, or “faster and cheaper” quietly becomes “whichever model was cheapest this hour.”
• Re-crawl on change, not on a timer. Connectors that lean on change-data-capture where the source supports it keep the graph and the index from drifting stale between full syncs.
• Log what was retrieved, scoped by entitlement, for every answer. When an answer is wrong or over-broad, the fastest diagnosis is knowing exactly what the graph and the index handed the model.
• Put meaning in the graph once, not in ten agent prompts. A definition copied into every agent’s context is a definition that will eventually disagree with itself; the ontology is the one place it’s allowed to live.
• Treat decision memory as a queryable store, not a chat transcript. It only compounds if the next run can ask “which path worked for this segment before” — not just replay what was said.

Key Questions

Q1) What is an enterprise graph, and how is it different from a knowledge graph?

A knowledge graph on its own captures company-wide relationships between content, systems workflows, and more. An enterprise graph goes further as it covers:

• a business ontology: the concepts, processes and policy behind that graph, mapped to real systems
• a personal graph per persona: their projects, collaborators and work style
• decision memory: retrieval and ranking are personalized, and every agent run leaves the next one further along, without maintaining a separate profile system or a separate log store alongside the graph.

Q2) What does an agentic search model actually do?

It is a small model, typically trained with reinforcement learning, that runs before the frontier model, not instead of it:

1. Executes a short tool plan over a fixed toolset—enterprise search, employee search, web search
2. Hands off to the frontier model with retrieved context already in place
3. Removes the frontier model’s tool-calling loop from the context window

That handoff is what removes a multi-turn loop, cutting latency roughly in half and token spend to a quarter—measured in production deployments of this pattern.

Q3) Which vendors fit this architecture?

The architecture is deliberately vendor-neutral. Different components come from different vendors:

• Glean pairs a company-wide and personal knowledge graph with its own agentic search model and multi-model routing.
• Elastic covers the retrieval and hybrid-index layer with the Elasticsearch Relevance Engine and can be paired with an external orchestration and model layer.
• Microsoft grounds Copilot in the Microsoft Graph.
• Coveo and Vertex AI Search fit the retrieval and indexing layer similarly.
• Neo4j targets the enterprise graph layer itself—the business ontology, policy and decision memory—as a governed graph substrate other agents and tools query rather than duplicate.
• Palantir providing the enterprise graph and the entire end-to-end intelligence stack.

The graph, retrieval, routing and governance layers are separable. A real deployment can mix a graph-and-assistant vendor with a different retrieval engine underneath it.

Q4) Why not just give every employee direct access to one frontier model?

Because personalization, entitlement and cost control all live below the model, not inside it:

• A frontier model with no graph cannot know what an employee is permitted to see
• It cannot apply a standing personalization signal without re-supplying it every call
• It offers no consistent way to cap cost or restrict which model runs for which team

The graph and the governance layer are what turn “an LLM with a system prompt” into an enterprise-safe assistant.

Q5) How is this different from a BI semantic layer, or a generic “context layer”?

Both feed a model better material. Neither governs what the enterprise means.

BI semantic layers (Looker, dbt, Power BI) pin down how a metric is computed—“revenue” is the same number everywhere. But a metric is an atom with no idea it connects to a customer, subscription or renewal date.

Generic context layers pull documents in alongside metrics. It is enrichment: the model sees more, but the organization has not agreed what things mean.

The enterprise graph here is executable, not definitional. It:

• Interprets intent against a governed model of the business
• Selects authoritative sources
• Enforces policy
• Captures the trace of how it got there

Instead of just handing back a number or a paragraph.

Q6) Who owns the graph once it exists?

Like data mesh for data: federated, not centralized.

• Domain teams own their own corner of meaning end to end. Only the finance team can say what “active customer” means in a P&L context—and live with the consequences if they get it wrong.
• Platform team keeps the shared graph reliable, queryable and versioned.
• Cross-domain governance sets what’s authoritative and where risk boundaries sit, enforced computationally at query time rather than renegotiated in a monthly meeting.

Federated ownership scales. A central team defining all meaning for all domains creates a bottleneck and a single point of failure.

Q7) What are the key Architecture Decisions (ADRs)?

• Two-stage reasoning over single-model loop. A small, fixed-plan agentic search model executes retrieval first, then hands off to the frontier model. This trades model versatility for predictable latency and token efficiency—roughly half and a quarter in production.
• Enterprise graph defined once, read everywhere. Meaning (ontology, entitlements, decision memory) lives in one governed layer. Every agent, search call, and ranking decision reads from it. No copies scattered into agent prompts.
• Entitlement filtering before ranking, not after. Filter by access at retrieval time, before the model ever sees a document. Filtering after generation does not reliably keep secrets out of the answer.
• Fixed tool plan for the agentic search model. The efficiency case collapses if the model is allowed to freelance. A short, bounded plan over known tools is what makes latency and token savings predictable.
• Personal graph as a signal, not a profile. Ranking and relevance shift based on the personal graph. It does not silently expand what an employee can retrieve. Personalization and entitlement are separate mechanisms.
• Model routing under policy, not by default. Automatic model selection needs an admin-set ceiling on cost and capability. Without it, “faster and cheaper” quietly becomes “whichever model was cheapest this hour.”
• Federated graph ownership, not centralized. Domain teams own their corner of the ontology end to end. A platform team keeps it reliable and versioned. Cross-domain governance sets boundaries, enforced computationally at query time.
• Vendor-neutral component layers. Graph, retrieval, routing and governance layers are separable. A deployment can pair a graph-and-assistant vendor with a different retrieval engine, or swap model providers without rewriting integrations.
• Decision memory as queryable store, not chat transcript. Every agent run leaves a trace. The next run asks “which approach worked for this segment before”—not just replay what was said. This compounds the value of repeated work.

References

This architecture generalizes the pattern described in:

• Glean’s third-generation AI Assistant and Enterprise Graph announcement
• Glean’s Waldo agentic search model launch, alongside the model-routing discussion in Latent Space’s interview with Glean.
• The business ontology, decision memory and shared-substrate framing draw on Neo4j’s enterprise knowledge layer for agentic AI.
• Palantir’s enterprise graph approach couples the Ontology as a semantic and kinetic digital twin with the Context Graph as a decision-aware reasoning substrate, transforming static data into a living operational graph where humans and AI agents can execute governed, closed-loop actions.