As enterprises rush to push generative AI into production, a familiar bottleneck has emerged: the infrastructure itself. Engineering teams are increasingly bogged down operating sprawling AI setups, managing costs, tracing errors, and configuring gateways, rather than actually building new applications.
Today TrueFoundry wants to fix that with a simple proposition: What if you could just ask your infrastructure what’s wrong?
Today, the company announced the launch of Ask TFY, a conversational interface built directly into its AI Gateway. Billed as a unified control room for engineering and operations teams, the new feature allows developers to query configurations, troubleshoot production bugs, and manage their AI infrastructure using natural language.
Untangling the LLMOps Web
As organizations mature past the pilot phase of their AI deployments, they are running into severe operational friction. Managing production AI today means manually cross-referencing guardrail configurations, monitoring multiple foundation model providers, tracking sprawling compute costs, and debugging application failures across highly siloed systems.
Often, the most senior engineers are the ones spending the most time hunting down critical operational data.
“Managing enterprise AI infrastructure at scale requires teams to cross-reference configurations, trace errors, and optimize costs across systems that weren’t built to talk to each other,” TrueFoundry co-founder and CEO Nikunj Bajaj said in a statement. “Ask TFY collapses that complexity into a single conversational interface so teams can surface what they need and act on it immediately.”
How Ask TFY Works
Instead of forcing developers to click through endless dashboards, Ask TFY acts as a specialized AI agent for the platform itself. The interface draws on three distinct knowledge bases to formulate its answers:
- Live Operational Data: Real-time AI Gateway configuration and trace data, including model routing, MCP (Model Context Protocol) servers, guardrail policies, budget rules, and request-level telemetry.
- Platform Documentation: TrueFoundry’s internal library of guides and best practices.
- Web Search: External queries for generalized troubleshooting and up-to-date developer context.
The resulting output isn’t just text. TrueFoundry says Ask TFY can dynamically render charts and UI elements, highlight system anomalies, and generate YAML code for configuration changes.
Crucially, the tool leans into agentic action rather than just passive observation. If Ask TFY recommends a configuration tweak, such as adjusting a rate limit, it can apply those changes directly upon the user’s request without requiring a manual handoff to another screen.
Key use cases highlighted by the company include:
- Security Audits: Asking “Which MCP servers are running without a guardrail?” instantly audits tool access across connected servers and renders the findings in a visual graph.
- Cost Optimization: Asking to “set quotas by developer, application, or model” prompts the tool to generate budget configuration recommendations based on live usage telemetry.
- Performance Tuning: Querying prompt cache utilization to find exactly where caching is being wasted, complete with a ready-to-deploy fix.
- Root Cause Analysis: Diagnosing recurring application failures and tracing the source of the error, whether it stems from the foundation model, the cloud provider, or the application layer itself.
Building the Central Control Plane
Ask TFY is the latest in a rapid cadence of feature rollouts from TrueFoundry this year, signaling the startup’s ambition to be the definitive “command center” for enterprise AI.
Last month, the company launched Agent Gateway, a control plane specifically for governing AI agents across various frameworks. Earlier in January, they rolled out TrueFailover, a resilience tool designed to keep AI workloads running seamlessly during major model, region, or provider outages.
The strategy seems to be resonating with heavy hitters. The platform is already being used at scale by enterprises including NetApp, Innovaccer, and Siemens Healthineers. According to TrueFoundry, its AI Gateway now processes over 1 trillion tokens per day, while its deployment platform manages over 1,000 clusters.
As the “LLMOps” space grows increasingly crowded, the winners will likely be the platforms that can abstract away the highest amount of toil. By turning complex infrastructure management into a simple chat prompt, TrueFoundry is betting that the best UI for managing AI is AI itself.
