Deepak Kaul has spent more than two decades designing and delivering mission-critical software across hospitality, banking, insurance, telecom, and government. His work spans large-scale digital modernization, AI, cybersecurity, cloud-native architecture, and adaptive enterprise systems, including a patented AI audio-analytics surveillance platform and additional patented work in cybersecurity and intelligent systems.
Alongside his engineering work, Kaul has built a substantial research profile, with more than 16 published papers across AI, cybersecurity, blockchain, and cloud-native microservices, as well as extensive peer-review experience for technical journals. He is also an IEEE Senior Member and has served as a judge and evaluator for industry awards and professional fellowships.
In this interview with The AI Journal, Kaul discusses what separates an AI-agent prototype from a production-ready system, why identity and permissions become more complex as agents gain autonomy, how enterprises should design for failure and rollback, and where human oversight still needs to remain firmly in place.
1. What is the biggest difference between building an AI-agent prototype that works in a controlled environment and deploying one safely across a large enterprise?
To me, the difference is operational accountability. Across travel, banking, and public-sector platforms, I have seen how a technically correct action can still cause harm when it reaches a stale record, a failed integration, or the wrong approval path. A production agent therefore needs its own identity, tightly bounded authority, reconciliation and rollback paths, monitoring, and a named business owner. A prototype proves capability; production proves that the organization can manage the consequences.
2. As AI agents gain the ability to take actions across multiple business systems, how should organizations rethink identity, authentication, and access controls?
Agents should never inherit a person’s broad credentials. I prefer a distinct workload identity, short-lived tokens, and the minimum authority needed for one task. Every action must establish three things: which agent acted, which human or business process authorized it, and what scope applied to that run. That separation makes forensic tracing, revocation, and least privilege practical, especially in enterprises where access accumulates over years.
3. What are the most important permission boundaries companies should establish before allowing AI agents to interact with sensitive data, APIs, or operational systems?
I start with reversibility. Reading a record, proposing a change, submitting it, and making an irreversible change are different risk levels, even when they use the same API. Controls must reach the tool, parameter, and record, not stop at the server. I also set per-run limits for records, spend, retries, and external calls. In enterprise architecture, the strongest control boundary is often closest to the business action, not the infrastructure layer.
4. How should enterprises approach observability and auditability so they can understand not only what an AI agent did, but why it took a particular action?
Auditability is not a polished explanation after an action. It is an evidence trail: the relevant context sources, policy evaluated, permissions, tool and schema version, parameters, result, and downstream correlation ID. This is familiar distributed-systems discipline, but agents need stronger linkage because their plans can change between steps. I would combine consistent tracing with immutable event records, while protecting sensitive content. A model’s explanation may help an operator; the verifiable evidence is the record of controls and actions.
5. Model reliability is only one part of the risk. What happens when an AI agent makes the correct decision but a downstream system, API, or integration fails?
This is classic distributed-systems engineering for a new kind of client. A timeout may mean a write succeeded but the caller did not receive the confirmation. If an agent retries creatively, duplicate reservations, refunds, tasks, or customer messages can appear. I design for an ‘unknown’ state, idempotency keys before every write, retry budgets, circuit breakers, and reconciliation with the system of record. The agent should choose from approved recovery paths, not invent a compensating action at runtime.
6. You have worked with AI-agent and Model Context Protocol-based systems. What opportunities does MCP create for enterprise AI, and what new security or governance challenges does it introduce?
MCP can make enterprise integrations more reusable and governable. Instead of rebuilding connections for every model or workflow, teams can expose controlled tools through consistent contracts. The benefit is not just speed; it is disciplined integration. Each server and tool definition should be inventoried, versioned, reviewed, and tested as a privileged interface. Organizations must validate authorization on every request, limit tool scopes, monitor invocation patterns, protect credentials, and re-approve material tool changes.
7. Based on your work across AI, cybersecurity, cloud-native systems, and intelligent automation, where should human oversight remain mandatory even as AI agents become more autonomous?
Human oversight should remain mandatory when an action is irreversible, affects a customer’s money, access, safety, or record, changes production systems, or falls outside conditions the agent has been tested against. In large enterprise transformations, I have seen that a fast automated action can be more damaging than a slow manual error because it scales instantly. Oversight must be meaningful: the reviewer needs a clear impact summary and the power to stop or reverse the action.
8. What architectural and governance practices do you believe will separate organizations that successfully deploy AI agents at scale from those that remain stuck in the prototype stage?
Organizations that scale agents will treat them as products and operational systems, not demonstrations. They will begin with a narrow outcome, a reliable system of record, named ownership, measurable success criteria, testing against realistic failures, and a rollback plan. Autonomy should progress gradually, from recommendation to supervised action to tightly bounded automation, as evidence accumulates. My experience modernizing large enterprise platforms has reinforced a simple lesson: identity, observability, resilience, and accountability must be designed in at the start, not bolted on later.
