From AI Features to AI Infrastructure: Ankit Roy on Building Intelligence That Scales

Ankit Roy has spent more than 15 years working across product management, software engineering, machine learning, experimentation, trust and safety, and enterprise technology. Over the course of his career, he has led the development of AI and machine learning systems spanning anomaly detection, causal inference, natural language processing, large language models, forecasting, privacy focused measurement, and experimentation, with several of those platforms ultimately adopted across multiple teams and product areas.

His work has included building reusable machine learning frameworks, scaling anomaly detection across engineering and operational organizations, developing causal inference systems used to measure product impact, and helping apply AI to trust and safety programs operating at massive consumer scale. He has also worked on systems designed to improve marketing measurement, knowledge discovery, product experimentation, and decision making, giving him a perspective that extends well beyond any single AI model or use case.

That experience has shaped Roy’s view that the next phase of AI product development will depend less on adding isolated intelligent features and more on building reusable infrastructure that can support many products, teams, and workflows. For him, the distinction comes down to architecture, ownership, shared data models, governance, and whether an AI capability becomes something an organization can improve once and apply repeatedly.

In this interview with AI Journal, Roy discusses what separates an AI feature from true AI infrastructure, why product teams often struggle to build reusable systems, how closed loop AI changes the way teams should think about risk and human oversight, and what will distinguish companies that simply use AI from those that build it deeply into the way their products operate.

You’ve argued that most AI investments inside products are really features rather than infrastructure. What distinguishes the two in practice, and why does that distinction matter for long-term competitive advantage?

A feature is built for a specific workflow, within the context of that one product. The model is trained and shaped around that use case, so the solution stays tied to it.

Infrastructure is something different. It’s more like a framework or a library that can be applied to similar workflows across different product areas. A good example is a summarization module. Different teams, in totally different product areas, can plug into the same underlying system. When you make improvements, you make them once, in one place, and every team benefits from it. That’s really the advantage and you’re not rebuilding the same solution five times across five teams and maintaining it separately. It also makes your solution look coherent across products. 

Why do so many product teams believe they are building AI infrastructure when they are actually adding intelligence on top of an existing workflow? What are the most common design choices that lead them in that direction?

It really comes down to how they’re building the models and what data they’re using. Most teams are focused on solving the problem in front of them, the one specific to their own product, so everything gets built around that context from the start.

There’s also a time pressure piece to it. Building something reusable takes longer than building something that just works for your immediate case, and most teams don’t have the incentive or the culture that rewards that extra investment. You’re measured on shipping your feature, not on whether three other teams can use what you built.

And honestly, there’s an experience gap too. Generalizing a workflow so it becomes a common solution across different use cases is a specific skill. It means recognizing the pattern underneath your problem, not just solving the problem itself. A lot of teams haven’t had the exposure to do that, so they don’t even see the option is there.

In your work across anomaly detection, causal inference, NLP, LLMs, and experimentation, what have you learned about designing AI systems that can move from identifying a signal to taking action without requiring a human at every step?

At the heart of it, it comes down to deeply understanding the problem space. That means spending real time with users, talking to experts, and looking at how the problem is solved outside your own industry or company. You can’t automate something well if you don’t actually understand why a human is doing it a certain way in the first place.

That understanding is what gives you a sense of the risks and the boundaries. Once you know where things can go wrong, you can set up the right guardrails. In practice, this could mean only letting the system act on its own above a certain confidence level, routing anything below that to a person. It could also mean starting with the system just recommending an action, then letting it act on a small, high-confidence instances, and expanding from there once that’s proven out. And you need a way to catch when the system starts drifting, so you’re not relying on it quietly getting worse without anyone noticing.

What architectural decisions need to be made early if a team wants AI to become part of the operating infrastructure of a product rather than remain a standalone feature or decision-support tool?

The first thing is identifying the right set of stakeholders early. You need to find the teams who will actually use this and agree on an ownership model up front, is this something a platform team owns, or does it end up as a side project inside one product team. That decision early on really determines whether it stays reusable or not.

Then you need to agree on the right data and schema before you even start building models. If every team is feeding in data differently or expects different outputs, you end up rebuilding the thing for every use case anyway.

The other piece is defining the core problem at the most granular and abstracted level you can. You want a foundation that different teams can build on top of and add their own context to as they customize it for their use case, instead of a system that only works for the first team that built it.

You’ve built machine learning platforms that were adopted across multiple teams and use cases. What makes an AI system reusable and scalable across an organization rather than something that has to be rebuilt for every individual application?

A big part of it starts even before you build anything, involving other teams early in the brainstorming process. You want to identify whether this is actually a common challenge across teams, not just something you’re assuming based on your own problem. That early involvement is also what tells you whether people will actually want to adopt it later, versus just building their own version anyway.

Like I mentioned before, you want to build the most granular, foundational version possible, the nuts and bolts, so other teams can customize on top of it instead of it being too specific to work for anyone else.

How you develop it matters too. Having something closer to an open source contribution model, where other teams can actually contribute back to it rather than just consume it, builds a different kind of culture around the platform. It becomes something people help shape, not just something handed to them

Closed-loop AI systems introduce a different level of risk because the system is not simply recommending an action; it may be taking one. How should teams think about confidence thresholds, safeguards, escalation paths, and human intervention when designing that kind of infrastructure?

The first step is identifying and categorizing the types of actions the system can take. A lot of actions are reversible, and those are usually safer to automate sooner. The irreversible ones need to be routed properly based on how confident the model actually is in that case.

I don’t think the goal should ever be full automation for its own sake. The real goal is automating as much as sensibly makes sense, enough to actually improve productivity, without taking on risk you don’t need to take on.

Testing quality matters a lot here too, and capturing human feedback along the way so you can keep improving the model where it’s needed. It’s really about identifying the right boundaries and risk tolerance for each category of task. This isn’t a binary switch, it’s more like a maze with a lot of different scenarios to handle. But once you’ve set those boundaries properly, it does end up improving overall productivity

Fraud detection, recommendation systems, engagement measurement, and marketing optimization all involve different types of signals and decisions. What principles remain consistent when designing AI infrastructure across such different domains?

Ultimately it comes down to the outcome you’re trying to drive, and what goes into the system along with its surrounding context. That part doesn’t really change from domain to domain. You still need ground truth, and you still need proper testing with human evaluation where it’s feasible.

Having the right guardrails matters just as much across every domain too. It’s not just about optimizing for the positive outcome, it’s about actively preventing the bad ones, because that’s really what builds trust in the system over time. Also, you need to find the right things to measure, along with a process to keep measuring and improving as the scenarios evolve. Signals change, behavior changes, so what you’re tracking can’t stay static either. 

As generative AI makes it easier than ever to add intelligent features to existing products, do you think the gap between companies that use AI and companies that build their products around AI infrastructure will become more important? What will separate the two over the next several years?

I think AI is going to become table stakes pretty quickly, and model capabilities will start to converge, the same way it’s played out with most other platforms before it. So the real value for a product was never just having AI, it’s still about solving the core user problem, with AI acting as a helper on top of that.

What will actually differentiate companies is how well it gets baked into the offering, and how contextualized it is to the specific problem you’re solving. Proprietary data is a big piece of that too, having access to something your competition simply doesn’t have.

The other differentiator is how safely you operationalize it. Are you actually keeping user trust, or are you just adding noise to the product. And how you improve the system over time, how you measure it, and whether you’re getting better at it while everyone else is still catching up on the basics