Why enterprise AI teams should stop building agents and start building agent platforms
On a whiteboard, an AI agent looks like a clean equation:
Agent = Prompt + Tools + Model
Three terms. Elegant. Ship it by Friday.
Then you actually build one for production. The equation grows:
Agent = Prompt + Tools + Model + Session Store + Streaming + State Management + Context Compression + Memory + File Upload + Code Execution + Tracing + Auth + Stop/Resume + Conversation Editing + Multi-Agent Handoff + Observability + Eval + Security Review + …
That trailing ellipsis is where engineering budgets go to die. Every team that sets out to build an agent discovers the same uncomfortable truth: the three terms that create value (prompt, tools, model) account for perhaps twenty percent of the total effort. The other eighty percent is infrastructure. And it is roughly the same infrastructure regardless of what the agent actually does.
After leading the design and build of a shared Agent Platform at ThoughtSpot, I have come to frame this as a simple thesis: Prompt + Tools = Value. Everything else is infra. The question is whether you build that “everything else” once, or once per agent.
The Replication Problem
Consider what happens when an enterprise decides to build not one agent, but three or four. Each serves a different function, perhaps one customer-facing, one internal, one specialised for a particular domain. Each team begins from scratch: standing up a conversation store, wiring up tracing, building streaming APIs, handling file uploads, implementing stop-and-resume logic, managing LLM context windows, and figuring out memory.
The result is predictable. Four teams solve the same twelve infrastructure problems in four slightly different ways. Bugs are discovered independently. Security reviews happen four times. Observability is inconsistent. And when leadership asks “can Agent A hand off to Agent B?” the answer is a painful silence, because the agents share no common substrate.
This is the replication problem. It is not a theoretical concern. It is the dominant cost centre in enterprise agentic AI today.
Separating Value from Plumbing
The insight that changed our approach was a clear separation of concerns. An agent’s unique value lies in exactly two things: the prompt that encodes its domain expertise, and the tools it can invoke to act on the world. A support agent’s value is in knowing how to interpret customer intent and call the right resolution tools. A planning agent’s value is in understanding context and orchestrating the right sequence of actions.
Everything else (session lifecycle, streaming event delivery, state persistence, context compression, tracing, memory, authentication, file handling, code execution sandboxes) is shared infrastructure. It does not vary meaningfully between agents. It should be built once, operated centrally, and consumed as a service.
This is what we mean by an Agent Platform. Not a framework. Not an SDK. A runtime.
What a Shared Agent Runtime Looks Like
At its core, the platform provides a Base Agent Service that any product team can configure declaratively. An agent definition is a configuration artifact: a system prompt template, a list of tool servers adhering to the Model Context Protocol, sub-agent references, and policies for context compression. The team writes this configuration and deploys. The platform handles the rest.
What the Platform Handles
Session persistence comes with resumability, checkpointing, and TTL policies. If a user leaves mid-conversation and returns later, the session is intact. Tool calls follow MCP, so every tool (internal or external) plugs in the same way, with results streamed back via SSE and state updates managed as metadata. Memory operates across user, organisation, and tenant scopes with typed knowledge categories (recipes, facts, preferences, rules, dictionary entries), generated after each invocation and retrievable by any agent as a searchable tool. Observability is native: every invocation emits OpenTelemetry traces automatically, with no per-agent instrumentation effort. No agent-specific code is required for any of these capabilities.
The Economics of Platform Thinking
The payoff is not subtle. When a new agent needs to be built, the team writes a prompt, builds its tools, and deploys a configuration. They do not provision infrastructure. They do not implement streaming. They do not build a conversation store. They do not negotiate with the security team about a new authentication flow, because the platform already handles cross-cloud identity federation with workload identity and short-lived tokens.
The marginal cost of the next agent drops dramatically. More importantly, multi-agent orchestration becomes feasible, because agents share a common session model and can be composed as sub-agents or invoked as tools by other agents.
What This Means for the Industry
The agentic AI ecosystem is maturing rapidly, and the frameworks (Google ADK, OpenAI Agent SDK, AWS Bedrock AgentCore, LangGraph) are converging on similar primitives. The differentiator for enterprises will not be which framework they choose. It will be whether they treat agentic infrastructure as a shared platform concern or let it fragment across product teams.
My recommendation to engineering leaders evaluating their agentic strategy: resist the temptation to let the first agent team build a bespoke stack. Invest early in the shared runtime. Define the boundary between what is “agent-specific” and what is “platform.” The line is clearer than you think.
Prompt + Tools = Value. Build the platform for everything else.
Ashish Shubham is an Engineering Fellow/VP at ThoughtSpot, where he leads AI infrastructure and developer platforms. He is an IEEE Senior Member and the author of Architecting AI Data Systems. He holds multiple AI analytics patents, and speaks regularly at IEEE, ACM, and industry conferences on agent architecture, enterprise AI, and developer experience.
