An AI/ML engineer and researcher explains why enterprise AI succeeds or fails on data quality, retrieval, observability, security, and human-centered design—not model size alone.
Chaitanya Devella has spent more than seven years building data, machine learning, search, recommendation, and generative AI systems. Today, as a Staff Machine Learning Engineer at Walmart Global Tech, his work focuses on enterprise AI platforms, retrieval-augmented generation, operational reliability, and agentic systems.
His research has been published at AAAI and IEEE Big Data. He recently presented EKRA, his work on reliable and incremental RAG ingestion for enterprise knowledge systems, at ICETICS 2026. He also contributes to the research community by reviewing technical manuscripts across multiple IEEE conferences.
In this interview, Devella explains why enterprise AI requires much more than a capable model. He discusses the engineering behind trustworthy RAG, the risks introduced by autonomous agents, the role of human judgment, and his ongoing research into personal finance assistants and the influence of frozen LLM knowledge on software ecosystems.
Your career began with large-scale data platforms before moving into search, recommendation systems, RAG, and agentic AI. What connects those stages?
The common theme is making complex information useful.
Early in my career, I worked on large data platforms, ingestion systems, distributed processing, and data governance. That experience taught me an important lesson: a model is only as dependable as the data and systems around it.
I later moved into enterprise search and recommendation systems. The problem changed from simply moving data to understanding what a person was looking for and connecting that person with the right information, job, learning resource, or next action.
RAG and agentic AI are a natural continuation of that journey. They bring together data engineering, information retrieval, machine learning, distributed systems, and user experience.
This is why I do not see enterprise AI as a model integration exercise. The model is one component. A useful production system also needs reliable knowledge, clear access controls, measurable quality, operational resilience, and a user experience that makes its limitations understandable.
What does it mean for enterprise AI to “earn trust”?
Trust is not created by a polished demonstration. It is built through repeated, predictable behavior.
A demonstration usually shows the system under ideal conditions. Production environments are different. Documents change. APIs fail. permissions are updated. Users ask vague questions. The model may retrieve incomplete evidence or select the wrong tool.
An enterprise AI system earns trust when users can understand where an answer came from, when the supporting information was last updated, and what the system does when it is uncertain.
From an engineering perspective, that means the system should be observable, traceable, permission-aware, and recoverable. Teams should be able to identify which source documents were retrieved, which model and prompt version were used, which tools were called, and where an error occurred.
A larger model may improve reasoning or language quality, but it cannot fix stale data, broken permissions, poor retrieval, or an unreliable workflow. Trust comes from the complete system.
Why is data ingestion such an important part of enterprise RAG?
RAG is often explained as a simple pipeline: split a document into chunks, generate embeddings, place them in a vector database, and retrieve them when a user asks a question.
That explanation is useful for learning, but real enterprise knowledge is constantly changing.
A document can be created, edited, removed, renamed, moved to another location, or assigned different access permissions. The same policy may appear in several systems with different versions. A file may be indexed successfully even when text extraction failed on several pages.
These are not small operational details. They directly affect answer quality.
A production ingestion system needs stable document identities, version tracking, content validation, duplicate detection, metadata enrichment, permission preservation, retry handling, deletion support, and clear relationships between source documents and their indexed chunks.
The pipeline also needs to answer basic questions quickly. Which documents failed? Which records are stale? How far behind is the index? Was a deleted document removed from retrieval? Are duplicate chunks affecting ranking?
When teams cannot answer those questions, the RAG system may sound confident while using incomplete or outdated information.
You recently presented EKRA, your work on reliable and incremental RAG ingestion. What problem does it address?
EKRA grew out of a practical issue: enterprise knowledge cannot be treated as a static collection of documents.
Many early RAG implementations focus on the initial ingestion process. They load an entire collection, create chunks and embeddings, and build an index. But the harder problem begins after the first load.
What happens when only one section of a document changes? What happens when metadata changes but the text does not? What happens when a source is removed? Reprocessing an entire repository after every modification is expensive and unnecessary, but updating only part of the repository creates consistency risks.
EKRA focuses on reliable, incremental ingestion. The idea is to identify the exact document or content segments affected by a change, update only those records, and maintain consistency between the source, metadata store, document chunks, embeddings, and vector index.
The approach also treats deletions and failed updates as first-class lifecycle events. A document that has been removed from the source should not continue appearing in search results because an old embedding survived in the vector database.
The larger point is that knowledge freshness is part of correctness. A well-written answer based on an outdated policy is still a wrong answer.
What should organizations measure before describing a RAG system as production-ready?
There is no single metric that captures RAG quality.
The first area is ingestion quality. Teams should measure extraction failures, duplicate content, missing metadata, indexing latency, stale records, permission mismatches, and document coverage.
The second area is retrieval quality. Useful measures include recall, precision, ranking quality, source authority, freshness, and whether the retrieved evidence actually supports the question.
The third area is generation quality. Teams need to evaluate groundedness, factual consistency, citation correctness, refusal behavior, completeness, and the system’s ability to communicate uncertainty.
Operational performance matters too. That includes latency percentiles, availability, cost per request, failure rates, retry behavior, recovery time, and the ability to roll back a model, prompt, embedding, or index change safely.
Finally, teams need to measure user outcomes. Did the system help the user complete a task? Did the user need to correct the answer? Was the response understandable? Did the system make the process faster without introducing unacceptable risk?
Industry guidance such as the NIST AI Risk Management Framework, the OWASP guidance for LLM applications, and ISO/IEC 42001 can help organizations structure these conversations. I view them as practical reference points rather than simple compliance checklists.
How does agentic AI change the risk profile of an enterprise system?
A traditional assistant usually returns information. An agent can take action.
It may query a database, call an internal service, create a record, initiate a workflow, modify a configuration, or communicate with another agent. That means an incorrect answer can become an incorrect action.
The goal should not be to avoid autonomy completely. It should be to introduce autonomy gradually and with clear boundaries.
I look for allowlisted tools, least-privilege access, narrowly scoped credentials, validated inputs and outputs, time and cost limits, complete audit trails, and approval gates for high-impact actions.
Actions should also be reversible whenever possible. If an agent creates or modifies something, the system should know how to roll it back or compensate for the change. Repeated requests should be idempotent so that a retry does not create duplicate records or transactions.
I believe an agent should earn autonomy through evidence. A team can begin with recommendation mode, observe the system’s behavior, introduce human approval, and expand autonomy only when measured performance supports it.
You contributed to AdaptJobRec, which was published at AAAI. What did that research teach you about designing agentic systems?
AdaptJobRec addressed an important trade-off in conversational recommendation systems.
Agentic reasoning can help with complex requests, but it can also increase latency. Not every user question requires memory processing, task decomposition, multiple tools, and several reasoning steps.
The research team developed a query-complexity mechanism that routes straightforward requests through a faster path while using more advanced planning for complex requests. In the reported evaluation, the approach reduced average response latency by as much as 53.3 percent against the evaluated baselines while improving recommendation accuracy.
The broader lesson is that intelligence should be applied selectively.
A production system does not need to use its most expensive model or deepest reasoning process for every request. Routing, specialization, caching, and deterministic services can often produce a better combination of quality, speed, and cost.
Good AI architecture is not about putting an LLM into every step. It is about knowing where probabilistic reasoning adds value and where conventional software is safer and more efficient.
Your IEEE Big Data research explored a hybrid vision-powered GUI agent. Why are hybrid architectures valuable?
GUI agents need to understand visual information, make decisions, and interact with interfaces. A large vision model can be useful at important decision points, but relying on it for every visual operation can increase cost and latency.
A hybrid architecture allows different components to handle the tasks they are best suited for. A vision-language model may interpret a complex screen or decide the next action, while lighter computer-vision methods can handle routine localization or repeated interactions.
This idea applies beyond GUI agents. In many enterprise AI systems, the best architecture is a combination of LLMs, retrieval systems, rules, APIs, classifiers, and deterministic validation.
The largest model is not automatically the right tool for every problem. A smaller model or conventional algorithm may be faster, easier to monitor, and more predictable for a narrow task.
The system becomes stronger when those components are orchestrated deliberately rather than forcing one model to do everything.
What role does operational excellence play in AI quality?
Users experience the whole product. They do not separate model quality from infrastructure quality.
A model may perform well during evaluation, but if the surrounding service times out, retrieves stale content, exceeds its budget, or fails during an important workflow, the user experiences it as a poor AI system.
Operational excellence should begin before deployment. Data, models, prompts, retrieval settings, and evaluation datasets should be versioned. Important changes should go through automated evaluation and controlled rollout. Teams should have clear service objectives, alert ownership, rollback procedures, and incident-review practices.
Monitoring should also go beyond CPU, memory, and HTTP errors. An AI platform should monitor retrieval quality, data freshness, groundedness, model refusals, tool failures, user corrections, quality drift, and cost.
Reliability is not separate from AI quality. In an enterprise environment, reliability is part of intelligence.
You also review research papers for technical conferences. How has that affected your work as an engineer and researcher?
Peer review has made me more critical of my own work.
A reviewer has to ask whether the research question matters, whether the proposed contribution is actually new, whether the baselines are fair, and whether the conclusions are supported by the evidence.
Those habits are useful in production engineering as well. Before accepting a result, I ask what was measured, what was excluded, which assumptions were made, and whether another explanation could produce the same outcome.
I have reviewed more than 50 manuscripts across technical conferences, including multiple IEEE conferences. The experience has exposed me to research in machine learning, distributed systems, cybersecurity, reinforcement learning, data quality, and applied AI.
I do not see reviewing as a title or a number. It is a responsibility. A good review should be technically honest, respectful to the authors, and specific enough to help them improve the work.
What are you researching now?
Two current projects examine very different aspects of applied AI.
The first is FincAI, a research prototype that explores how multimodal and language models can support personal finance workflows. It brings together receipt understanding, spending analysis, educational financial guidance, and recurring-transaction automation.
The important research questions are not limited to whether a model can read a receipt or produce advice. They include how to ground responses in verified financial data, maintain numerical consistency, protect sensitive information, handle automation failures, and keep a human involved when a model is uncertain.
FincAI is designed as an informational and educational assistant rather than a replacement for a qualified financial professional.
The second project studies what I call the Knowledge Wall. Large language models have a training cutoff, which means their internal knowledge represents software libraries available before that boundary more strongly than libraries released afterward.
I am studying whether that frozen knowledge can influence real-world software adoption. The work uses large-scale PyPI and GitHub data and causal-inference methods to examine whether libraries released on different sides of a major model cutoff follow different adoption patterns after AI coding assistants become widely used.
The broader question is important: when developers increasingly rely on the same models for recommendations and code generation, can the model’s knowledge boundary shape which technologies become visible and which remain overlooked?
Both projects are ongoing and are being prepared for peer-reviewed publication.
How does human-centered design fit into highly technical AI infrastructure?
Human-centered design is not only about the interface. It affects what a system optimizes, how it communicates uncertainty, who is represented in the data, and what happens when the system is wrong.
During graduate school, I collaborated with a professor on an iOS application connected to Still Surviving, a platform centered on stories from HIV/AIDS survivors.
My role was a collaborative engineering contribution rather than ownership of the overall project, but the experience stayed with me. It showed that technology can support dignity, representation, and community—not just productivity.
It also raised questions that remain relevant in AI. Are people given control over their information? Are we designing for users who may have different levels of technical confidence? Can the system explain what it is doing? Does it create value without reducing a person to a data point?
These questions matter even when the underlying work involves APIs, embeddings, access controls, or distributed systems. The infrastructure ultimately shapes a human experience.
What do you expect from the next phase of enterprise AI?
I expect enterprise AI to become less focused on one universal model and more focused on coordinated systems.
Organizations will use specialized models, retrieval layers, deterministic services, knowledge graphs, workflow engines, and governed agents. The competitive advantage will not come only from accessing a capable model. Many organizations can do that.
The advantage will come from reliable knowledge operations, effective evaluation, domain expertise, security, and the ability to improve systems safely over time.
Agent evaluation will also become more realistic. It will not be enough to show that an agent completed a task once. Teams will need to measure how it behaves over long-running workflows, how it handles conflicting instructions, how it recovers from failure, whether it respects permissions, and how effectively it works with people.
I also expect knowledge freshness to become an established engineering discipline. Organizations will need to know where an answer came from, when the source changed, whether the user was authorized to see it, and how quickly incorrect or outdated information can be removed.
The model may generate the answer, but the surrounding system determines whether that answer deserves to be trusted.
What advice would you give engineers who want to build meaningful careers in AI?
Do not build your identity around one framework or one model. Both will change.
Build durable skills in software engineering, data systems, statistics, security, evaluation, distributed computing, and the domain where you want to apply AI.
Learn to build an end-to-end system rather than stopping at a notebook demonstration. Understand what happens when data changes, an API fails, a user asks an unexpected question, or a model produces the wrong result with high confidence.
I would also encourage engineers to contribute to the wider technical community. Write about what you learn. Publish when you have a real research question and evidence. Review the work of others carefully. Share useful implementations when intellectual-property and privacy restrictions allow it.
Most importantly, stay close to the problem and the people affected by it. The strongest AI engineers are not necessarily the people who know the most model names. They are the people who know when AI is useful, when conventional engineering is better, and when human judgment must remain in control.
“A larger model may improve reasoning, but it cannot fix stale data, broken permissions, poor retrieval, or an unreliable workflow.”
About Chaitanya Devella
Chaitanya Devella is a Staff Machine Learning Engineer at Walmart Global Tech with more than seven years of experience in enterprise AI, machine learning systems, search, recommendation systems, retrieval-augmented generation, and agentic AI.
He has contributed to research published at AAAI and IEEE Big Data and recently presented EKRA, his work on reliable and incremental RAG ingestion for enterprise knowledge systems, at ICETICS 2026. He has also reviewed more than 50 manuscripts across technical conferences, including multiple IEEE conferences.
Devella holds master’s degrees in Computer Science and Artificial Intelligence and is currently pursuing an MBA. His research interests include reliable enterprise LLM systems, knowledge freshness, AI-assisted software development, trustworthy agent evaluation, and human-centered AI.
His professional contributions have received multiple organization-level Make the Difference awards and more than ten Bravo recognitions at Walmart.
Disclaimer
Chaitanya Devella is speaking in his personal capacity. The views expressed in this interview are his own and do not necessarily represent the views of Walmart. No confidential or proprietary information is discussed.
