For the past several years, artificial-intelligence progress has been described largely through the capabilities of individual models. Larger parameter counts, longer context windows, stronger reasoning, multimodal understanding, and more sophisticated post-training have pushed foundation models into areas that once required separate machine-learning systems.
That progress is real, but it is also changing the nature of the problem. As AI moves from answering questions to retrieving information, using tools, maintaining memory, selecting among models, and taking sequences of actions, the foundation model is becoming only one part of a much larger decision architecture.
That is why I believe the next major AI breakthrough will increasingly be a systems breakthrough.
The central challenge is no longer simply whether a model can produce a good answer in isolation. It is whether an AI system can obtain the right evidence, decide how much computation a task requires, choose the right tools, preserve useful context, recognize uncertainty, recover from intermediate errors, and remain reliable as the environment changes.
A highly capable model can still sit inside a weak system. If retrieval surfaces the wrong evidence, if routing sends a difficult request to an inadequate model, if memory preserves stale assumptions, or if a tool call changes the environment in an unexpected way, improving the underlying model may not solve the failure. The quality of the final outcome depends on how the components interact.
The Unit of Intelligence Is Expanding
Traditional machine learning encouraged us to think about intelligence primarily at the model level. We trained a model on data, evaluated its predictions, and improved its performance against a well-defined objective. The production environment could be complicated, but the core object of study was still the model.
Modern AI systems are structurally different. A single request may pass through query interpretation, retrieval, reranking, context construction, a language model, a tool-selection layer, an external API, memory, and another model before a final result appears. In an agentic workflow, that process may repeat several times.
The practical consequence is that system behavior cannot be inferred from model quality alone. A frontier model with weak retrieval can be less useful than a smaller model connected to authoritative, well-ranked information. A sophisticated planner can fail because an external tool returns malformed data. A long-context model can make worse decisions if its context is filled with obsolete or contradictory information.
This shifts the research question from “How capable is the model?” to “How capable is the complete decision process?”
That distinction will matter more as AI systems become embedded in enterprise workflows, commerce, software operations, research, and other environments where the system does not merely generate text but influences what happens next.
Retrieval Is Becoming Part of Reasoning
Retrieval-Augmented Generation is often presented as a way to give a language model access to information outside its training data. In practice, retrieval is becoming much more fundamental than that description suggests.
Once a model depends on external information to answer a question or make a decision, the retrieval pipeline becomes part of the reasoning process itself. The system has to interpret the request, search the right collection, rank candidate evidence, distinguish authoritative sources from incidental matches, decide how much information to place in context, and sometimes determine whether additional retrieval is necessary.
A language model can reason correctly over the evidence it receives and still produce the wrong answer because the relevant evidence never reached it. That is not a model failure in the conventional sense. It is a system failure.
This is why RAG evaluation cannot stop at answer quality. Retrieval recall matters because the right evidence must enter the candidate set. Ranking quality matters because relevant passages need to appear near the top. Provenance matters because the system should know where information originated. Freshness matters because a semantically relevant source may no longer be valid. Groundedness matters because the final response should actually reflect the retrieved evidence.
The next step is even more interesting: retrieval that becomes adaptive. Instead of performing one search and immediately generating an answer, a stronger system should be able to recognize that its current evidence is incomplete, reformulate the query, compare conflicting sources, and retrieve again before acting.
At that point, retrieval is no longer a static lookup operation. It becomes a decision policy.
Routing Will Determine How Much Intelligence a Task Receives
Another systems problem is emerging from the rapid expansion of available models. We now have large general-purpose models, smaller efficient models, domain-specialized systems, multimodal models, embedding models, rerankers, and task-specific components. Sending every request to the largest model is simple, but it is rarely the most efficient architecture.
Different tasks require different amounts of reasoning. A straightforward extraction request may be handled well by a smaller model. A difficult planning problem may require a stronger reasoning model. A domain-specific task may benefit from a specialized component. Some requests may require retrieval first; others may not.
This turns model routing into a core optimization problem. The router has to estimate task complexity before it knows the final answer. It has to balance expected quality against latency, computational cost, and failure risk. It also has to know when a smaller model is sufficient and when the task should be escalated.
This is one reason complexity-aware routing and knowledge distillation are important areas of AI systems research. Distillation can transfer selected behaviors from large models into smaller ones, while routing can determine when those efficient models should be used.
The long-term architecture is unlikely to be one model handling everything. It is more likely to be a hierarchy in which the system dynamically allocates intelligence based on the problem in front of it.
That is a meaningful shift in how we think about scaling. Instead of asking only how powerful a model can become, we also need to ask how intelligently a system can allocate computational power.
Agentic AI Introduces Sequential Risk
Agentic AI makes the systems problem more difficult because actions change the conditions under which later decisions are made.
A chatbot generally produces an output. An agent may retrieve information, interpret it, choose a tool, execute an action, observe the result, update its internal state, and then determine what to do next. The output of one step becomes part of the input to another.
That creates a fundamentally different error structure.
A retrieval mistake can produce an incorrect plan. The incorrect plan can cause the wrong tool to be selected. The result of that tool call can then become new evidence for the next step. What began as a small upstream error can propagate through the entire workflow.
This is closely related to sequential decision-making and reinforcement learning, where an action changes the state from which the next action is selected. A decision cannot always be evaluated in isolation because its consequences extend beyond the current step.
For agentic systems, that means reliability should be evaluated across trajectories, not just final answers. We need to understand whether the agent selected the correct intermediate actions, whether it used tools appropriately, whether it recognized when a step failed, whether it recovered, and whether it knew when to stop.
This becomes even more important when actions are consequential. Generating an incorrect paragraph and making an incorrect external change are not equivalent failures. As systems gain greater autonomy, authorization boundaries, recovery mechanisms, and abstention behavior become part of the intelligence architecture itself.
Benchmarks Need to Measure the Path, Not Only the Outcome
Machine-learning benchmarks have been enormously useful because they allow researchers to compare models on standardized tasks. But they are often designed around final-output quality, and that is not enough for a multi-component AI system.
A system can produce the correct answer through a poor process. It might retrieve unreliable evidence, invoke an unnecessary tool, or reach the right result only because two errors happened to cancel each other out. Conversely, an agent may follow a sound process and fail because a third-party service is unavailable.
Those cases should not be treated as the same type of failure.
System evaluation therefore needs multiple layers. Retrieval should be evaluated independently from generation. Routing should be assessed for both quality and efficiency. Tool use should be evaluated for selection, argument construction, execution, and recovery. Memory should be tested for relevance and temporal validity. Agent trajectories should be examined for unnecessary steps, repeated errors, and the ability to recognize insufficient evidence.
End-to-end success still matters, but it should sit on top of these intermediate measurements rather than replace them.
This also changes observability requirements in production. Engineers need to reconstruct what the system actually did: what it retrieved, what model it selected, which tools it invoked, what evidence changed the decision, and where uncertainty entered the workflow. Without that visibility, debugging an agentic system becomes far more difficult than debugging a conventional prediction pipeline.
AI Systems Need Stress Testing
One lesson I have carried from quantitative and model-risk work is that evaluating a system only under favorable conditions provides an incomplete picture of reliability.
Financial models are tested across changing scenarios because relationships that appear stable historically may behave differently under stress. AI systems increasingly need the same mindset.
A production AI system may face missing documents, conflicting evidence, distribution shifts, unavailable tools, outdated policies, ambiguous requests, unexpected user behavior, or inputs that are materially different from those seen during development. Those should not be treated as rare edge cases. They are part of the operating environment.
An effective AI stress test should deliberately make the system uncomfortable. Remove a critical source. Introduce two credible sources that disagree. Make an external tool fail. Supply stale information. Increase task ambiguity. Force the router to choose between a cheaper model and a more capable one. Observe whether the system asks for more evidence, escalates appropriately, recovers, or continues confidently in the wrong direction.
The important question is not only whether the system works when everything around it is working. It is how gracefully the system degrades when assumptions break.
That will become one of the defining characteristics of reliable agentic AI.
Memory Is More Than a Longer Context Window
Longer context windows are useful, but they are not the same thing as memory.
Persistent AI systems may accumulate interactions across days, months, or years. Simply placing all prior information into every prompt is computationally inefficient and can actively reduce quality by introducing irrelevant, stale, or contradictory information.
A useful memory architecture has to make at least two decisions: what should be remembered, and what should be recalled for the current task.
Both are difficult.
Some information is durable. Some is temporary. A preference expressed last year may have changed. A policy document may have been superseded. A prior inference may have been incorrect from the beginning. In other cases, the fact that something changed is itself important and should be preserved.
This means memory systems need concepts that ordinary semantic retrieval does not fully capture: recency, temporal validity, provenance, confidence, contradiction resolution, and forgetting.
An agent that remembers everything indiscriminately can become just as unreliable as an agent that remembers nothing.
The goal is not maximum memory. It is useful memory.
Multimodal AI Creates an Evidence-Arbitration Problem
The systems challenge becomes even more complex when information arrives through multiple modalities.
Real-world decisions may depend on text, structured data, images, video, audio, behavioral signals, and machine-generated metadata. Multimodal models have made impressive progress in representing these sources, but representation is only part of the problem.
A production system also has to decide what to believe when the evidence disagrees.
A text description may conflict with a structured record. A recent behavioral signal may be more useful than a historical preference. A primary document may deserve more authority than an automatically generated summary. An image may provide evidence that changes the interpretation of accompanying text.
This requires more than feature fusion. It requires evidence arbitration.
The system needs to account for source reliability, freshness, confidence, provenance, and task relevance. The most semantically similar information is not necessarily the most authoritative information.
As AI systems become more multimodal, this ability to reason about the quality of evidence—not simply its content—will become increasingly important.
Efficiency Will Become a Property of Intelligence
The economics of AI are also becoming a systems problem.
If every request uses the largest available model, the maximum context window, multiple retrieval passes, and several verification steps, quality may increase, but cost and latency can quickly become unacceptable at scale.
A mature AI architecture should spend computation where computation has the highest expected value.
That means combining several techniques rather than relying on one. Retrieval can provide external knowledge without encoding every fact in model parameters. Parameter-efficient adaptation can specialize behavior without retraining an entire model. Distillation can move useful capabilities into smaller models. Routing can reserve the most expensive reasoning for tasks that actually require it. Verification can be triggered selectively when uncertainty or consequence justifies the additional cost.
The deeper idea is adaptive computation.
An intelligent system should not only solve the problem. It should also make a reasonable decision about how much intelligence the problem deserves.
This is where efficiency and capability stop being separate engineering concerns. Efficient allocation of computation becomes part of system intelligence itself.
The Next Frontier Is Reliable Coordination
Foundation models will continue to improve. Scaling, multimodal learning, post-training, reasoning, and architectural innovation will keep expanding what individual models can do. But increasingly capable models will also expose the limits of weak system design more clearly.
The next generation of AI will need to know when its internal knowledge is insufficient and retrieve external evidence. It will need to distinguish current information from stale information, allocate the appropriate level of computation, choose tools safely, maintain useful memory, evaluate conflicting evidence, and recognize when uncertainty is too high to act autonomously.
Those are not peripheral engineering problems. They are increasingly central AI problems.
The field is therefore moving toward a broader definition of intelligence. A useful system is not merely one that can generate the next token with high probability. It is one that can assemble the right information, choose the right computational path, reason across multiple steps, adapt when the environment changes, and recover when part of the process fails.
The next breakthrough may not come from one model becoming dramatically better at everything. It may come from learning how to coordinate models, retrieval, memory, tools, and feedback into one coherent decision process—and making that process reliable enough to trust.
Author Bio
Chandana Dayapule is a machine-learning scientist whose work spans quantitative modeling, reinforcement learning, recommendation systems, knowledge retrieval, and applied AI systems.
