Sai Vishnu Kiran Bhyravajosyula has spent more than a decade designing the invisible engines that power the world’s largest digital platforms. Now a principal engineer at Atlassian, he has led major infrastructure initiatives at Salesforce, Confluent, Uber, and AWS, shaping the evolution of distributed systems that handle data at planetary scale. As AI transforms how systems are built, scaled, and maintained, Bhyravajosyula brings a unique perspective on where data infrastructure, automation, and reliability are heading next. In this conversation with AI Journal, he shares lessons from his career, insights into AI-driven scaling and observability, and his vision for the future of intelligent, adaptive infrastructure.
You’ve built large-scale distributed systems at Uber, Salesforce, and AWS. As systems evolve to integrate AI-driven components, what new challenges do you see in scaling both data infrastructure and model performance reliably?
As large-scale systems integrate AI-driven components, the nature of scalability is shifting from pure throughput and reliability to data freshness and managing dynamic systems.
On the data side, scaling now means maintaining consistent, high-quality, and traceable data flows that support continuous model training and real-time inference. This requires evolving data storage, real-time pipelines, and auditable systems that ensure feature consistency and minimize drift between training and production environments. For example, while I was at Salesforce, we noticed a high data increase in the Service Cloud’s conversations platform, which was attributed to the heavy use of the AI Agents. We had to come up with some intelligent ways to scale our data stores.
The next generation of scalable systems will involve creating resilient, self-correcting platforms where models, data, and distributed infrastructure evolve cohesively and reliably.
Serverless and event-driven architectures have transformed how developers think about scalability and cost. How do you see AI extending those principles, especially in areas like predictive scaling, adaptive resource allocation, and performance optimization?
In predictive scaling, AI models can forecast load patterns using historical telemetry, user behavior, and seasonality to preemptively allocate resources, reducing both latency and cold-start penalties. My patent, “model-based selection of network resources for which to accelerate delivery” has a similar idea where we predict the next resource to accelerate based on the customer usage patterns.
Today for adaptive resource allocation, we rely only on metrics like CPU/memory/io utilization for auto-scaling resources. With AI, we can continuously learn the deployment/usage patterns of the system and automatically tune compute, storage, and network configurations based on real-time performance signals. Additionally, I envision AI predicting the necessary configuration(s) for resources to optimize costs and provide options across multiple clouds.
In performance optimization, AI can analyze complex, high-dimensional metrics to detect anomalies, suggest architectural improvements, or even auto-remediate performance degradation before users notice.
At Uber, AWS, and Salesforce, you helped design systems that supported real-time responsiveness at a global scale. What lessons from those experiences apply to today’s world of distributed platforms that must handle vast, dynamic data flows?
Building real-time systems at a global scale taught me that availability, graceful degradation, and data locality are key to resilience. Building with failure and adaptation in mind is what truly enables systems to perform reliably at a global scale. At Uber and Salesforce, we relied on emitting real-time metrics, streaming platforms like Kafka, and asynchronous pre-processing orchestration systems like Temporal/Cadence that could ingest, route, and process high-velocity data near its source. These experiences shaped my belief that resilience and adaptability must be designed from the start, leveraging robust, proven infrastructure and self-observing pipelines to ensure systems respond effectively as data patterns and volumes evolve.
Edge computing is now essential for low-latency AI workloads, from personalization to real-time analytics. What design trade-offs do you see between deploying intelligence at the edge versus centralizing it in the cloud?
Deploying AI at the edge enables instant, context-aware responses, but adds complexity in managing updates, data drift, and hardware variability. Centralized cloud AI simplifies training, monitoring, and governance but introduces latency and network dependence. The best systems blend both: lightweight inference at the edge for speed, with centralized retraining and analytics in the cloud. This hybrid model strikes a balance between real-time performance and consistency, reliability, and continuous learning at scale.
Many organizations are now utilizing AI for automated observability, anomaly detection, and incident response. How realistic is the vision of “self-healing” infrastructure, and what limitations still stand in the way of truly autonomous operations?
The idea of “self-healing” infrastructure is compelling, but it is still a mix of reality and aspiration. For deterministic, well-understood failure modes, self-healing is already practical and widely deployed. Examples include: spotting CPU spikes, latency anomalies, or failed deployments, forecasting load patterns, and preemptively scaling clusters or cloud services.
The main issues I see with the current self-healing infra are:
- Limited Context: Lacks business context: For example, it might detect a drop in throughput, but cannot always decide if it’s acceptable or catastrophic.
- High false positives: AI models are only as good as the data they observe. Incomplete instrumentation, noisy logs, or delayed metrics can lead to false positives/negatives.
- Risk of automated fixes: Automated fixes can exacerbate issues. Restarting a service without understanding the root cause may lead to cascading failures, data loss, or regulatory violations.
- Compliance and Security: Autonomous changes may inadvertently violate compliance or data residency requirements.
The realistic vision of self-healing infrastructure today is partial autonomy. Over time, AI with deeper causal reasoning and contextual awareness may handle increasingly complex issues, but the current practical approach is AI-assisted operations combined with human-in-the-loop decision-making for critical or unfamiliar incidents.
You’ve worked extensively on multi-cloud reliability and data portability. In a world where AI depends on consistent, accessible data across platforms, how can companies design systems that are both performant and compliant with regional constraints?
Balancing performance, reliability, and compliance in a multi-cloud, AI-driven world requires a careful combination of architectural strategy, tooling, and governance. Companies need a “cloud-agnostic, region-aware AI fabric”—where AI workloads can seamlessly access the right data, in the right place, at the right time, without violating regulations. The key is abstraction, automation, and policy-driven architecture:
- Abstract data access from its location.
- Automate compliance and region-aware orchestration.
- Standardize formats and APIs to enable portability.
- Optimize compute and storage placement for performance.
- Continuously monitor and test both data quality and compliance.
Developer productivity remains a recurring theme in your work. How do you think cutting-edge developer tools, from intelligent debugging to code generation, will change how engineers build and operate distributed systems in the next few years?
AI-driven code generation is already becoming standard across the industry. It can automatically build the scaffolding for complex infrastructure and microservice endpoints, enabling engineers to spin up functional environments within minutes. For instance, an engineer could request a globally distributed bookkeeping service, and the AI would instantly produce the necessary CRUD APIs, Kubernetes manifests, Terraform scripts, and CI/CD pipelines.
Similarly, intelligent debugging agents are changing how teams diagnose and resolve issues in large-scale systems. By correlating logs across services and tracing distributed transactions, these tools can pinpoint the root cause of incidents such as time-outs or network bottlenecks. In one example, when a checkout process failed, the AI quickly identified thread pool exhaustion in a third-party payment service, saving hours of manual investigation and reducing downtime.
Looking forward, where do you see the intersection of AI, serverless, and distributed systems leading us? Will the next era of infrastructure be defined by autonomous orchestration, or by tighter collaboration between human engineers and machine intelligence?
AI, serverless, and distributed systems are converging to create infrastructure that thinks and adapts. The next era won’t be fully autonomous nor purely human-driven; it will be collaborative intelligence, where humans set direction and AI continuously optimizes for performance, cost, and resilience, enabling platforms that self-heal, self-tune, and respond dynamically to ever-changing data flows at a global scale. Systems can now predict workloads, auto-scale serverless functions, and intelligently route data. Some examples I can think of are:
- Self-optimizing serverless functions using AI to manage cold starts and resource scaling.
- Cost-optimizing resource provision across multi-cloud.
- Autonomous incident response where AI diagnoses and self-heals system failures in real-time.
- AI-assisted code generation for deploying new distributed services based on human-defined goals.
As AI reshapes the fabric of distributed computing, Bhyravajosyula’s perspective bridges the pragmatic and the forward-looking. His experience across multiple generations of cloud architecture offers a grounded view of how automation, reliability, and human ingenuity will coexist. The path he describes is of systems and people learning together, adapting, optimizing, and building the next wave of global infrastructure.
