In 2023, 23andMe suffered a credential stuffing attack that initially compromised 14,000 accounts. However, the impact was magnified by a governance failure. The company’s DNA Relatives feature let those compromised accounts access data on 6.9 million biological relatives, a 500x blast radius that came from a design choice granting accounts far wider access than any individual user had authorized.
By March 2025, the company had filed for Chapter 11. Four months later it was sold off for a fraction of its original worth.
23andMe is a cautionary tale. They built a system with overprivileged access and it cost them greatly. Without governance, AI agents can introduce the same risk.
Giving AI access to your company data is necessary for them to function, but it also expands the blast radius of any single compromise. As the 23andMe team discovered, the lack of governance was the reason the initial attack went from something contained to something catastrophic. The lack of governance cost them their customers’ trust and, consequently, their company. Governance risks are abstract until they are not.
According to the Thales 2025 Digital Trust Index, 82% of customers have abandoned a brand because of privacy concerns. Yet 83% of organizations plan to deploy agentic AI into their business functions this year, and only 31% feel fully equipped to secure those systems.
With more organizations deploying agents at scale without guardrails in place, data privacy exposure is inevitable. Consumers will respond accordingly with their wallets.
But it doesn’t have to be this way; businesses do not have to take on unnecessary risk while deploying AI. Taking that a step further, businesses also don’t have to slow down in order to reduce risk. Applying proper AI governance principles early actually accelerates AI adoption.
MCP raised the stakes for data governance
The widespread adoption of MCP has changed what an ungoverned AI deployment actually means. MCP has become the dominant standard for connecting AI agents to external tools, data sources, and enterprise systems. And the reason MCP spread so fast is simple: it enables powerful agentic systems that can actually do real work.
An agent connected via MCP to your CRM, your project management tools, and your internal knowledge base is powerful. But to deploy these workflows responsibly, companies must have guardrails. Without runtime policies, MCP can give agents unfettered access to data that should never reach an agent’s system, like PII or unconsented user data.
The failure modes are faster and harder to trace, especially for companies that don’t take observability seriously. Gartner predicts that by 2030, 50% of AI agent deployment failures will stem from insufficient governance runtime enforcement. When something like that goes wrong, assigning accountability requires knowing what the agent did, when, and with which data. Most organizations currently lack that visibility entirely.
In fact, only one in five organizations has a mature governance model for autonomous AI agents, according to a recent Deloitte survey. That number sits alongside aggressive deployment timelines in a way that should make enterprise leaders uncomfortable. The agents are already running. The frameworks that should be managing them, in most cases, are not.
Vendor reputation is not a governance model
One of the most dangerous assumptions in enterprise AI right now is that first-party MCP servers from reputable vendors are safe by default. This is not always the case.
Researchers found prompt injection vulnerabilities in both GitHub’s and Atlassian’s MCP servers, two of the most widely trusted implementations in the ecosystem. In both cases, researchers demonstrated that hiding malicious instructions inside ordinary content, a GitHub issue or a Jira support ticket, was enough to cause the model to execute those instructions with the internal user’s privileges.
This is the failure mode that gave us zero trust security. For years, corporate networks ran on a castle-and-moat model: once you were inside the firewall, you were trusted. Then breach after breach showed that attackers didn’t need to climb the wall when they could walk through the gate with stolen credentials or piggyback on a vendor who was already inside. Zero trust flipped the assumption. Verify every request, every time, regardless of where it came from. Check identity. Scope permissions. Treat nothing as safe just because it arrived from a familiar source.
MCP needs the same thinking. The GitHub server isn’t malicious. Neither is Atlassian’s. But the moment a company assumes “first-party vendor means safe to wire up unsupervised,” they’ve rebuilt the castle and moat on top of a protocol that hands an AI agent keys to their most sensitive systems. What matters is what controls sit between the data those servers return and the actions an agent can take with an internal user’s privileges. Without those controls, the consumer trust implications are significant.
Agents are creating the next customer trust crisis
A 2025 digital trust study found that 44% of consumers say transparency around how their data is used is the number one driver of brand trust.
The same research found that 77% of global consumers still do not fully understand how their data is collected and used, and a recent MIT insights report reinforced that AI-driven systems are amplifying this confusion rather than resolving it.
Consumers are increasingly aware that something is happening with their data; they just can’t see what, or why, or whether they ever consented to it. When that uncertainty tips into distrust, 82% walk away from using that product or brand.
The permission architecture, therefore, has to be in place before the agent acts. Doing so protects consumer data and consumer loyalty.
What solid AI governance looks like
Most governance conversations end with high-level principles that are hard to act on. Having deployed AI governance guardrails and policies for all different types of business, I prefer to give tangible steps teams can use to deploy AI, safely.
Here’s what solid AI governance looks like in practice:
- Get visibility before you scale. If you can’t answer which systems an agent touches and what data it acts on, you don’t have the foundation for governance. Audit logging and real-time observability across agent activity are now the baseline. MCP gateways have emerged specifically to give organizations centralized visibility and control over MCP-connected agents: what servers they’re connecting to, what tools they’re invoking, and what data is moving where.
- Treat consent as architecture, not disclosure. Privacy policies don’t travel with data into AI systems. Consent infrastructure needs to be embedded at the integration layer, specifying what agents can access, enforcing user preferences in real time across every system the agent touches. This is a meaningful shift from how most organizations currently handle consent, and it requires investment in the connective layer between consent management and AI tooling.
- Build cross-functional ownership with a named accountable party. Legal, security, and AI teams tend to operate independently, without clear ownership over who owns AI policies and enforcement. The result is accountability that belongs to everyone in theory and no one in practice. Defining who is accountable is critical to governance taking hold at companies pushing for AI innovation. I recommend the AI team integrate this as a core responsibility, since they’re the ones most able to understand the fast-evolving systems and plan accordingly.
- Enforce policies in real-time. Auditing what happened after an incident only surfaces problems once the consequences are already in motion. Policy enforcement, including RBAC, PII detection, token management, and allowlists for approved MCP servers and tools, needs to happen at runtime, before the agent acts.
The past decade forced organizations to acknowledge privacy. The next one will require them to design around it. The organizations that make that shift now, before the next trust incident lands in a headline, will be the ones that earn the right to scale AI, while also keeping the customers they spent years building.
The path to AI acceleration Is a governed one
The data clearly shows that companies are eager to move fast with AI. However, what many leaders might not realize is that the most innovative companies using AI rely on governance to move faster. When the proper governance guardrails are in place, teams can explore more freely and move more quickly. In this way, governance is an AI accelerator.
Cisco’s 2026 Data and Privacy Benchmark Study found that 99% of organizations that invested in privacy and data governance report measurable benefits, from faster innovation to stronger customer trust.
Organizations that can enforce runtime policies and see what their agents are doing are the ones that can confidently expand what those agents are allowed to do. They are also the companies that customers will continue to trust in the future.
AI systems with solid governance in place also just perform better. Gartner found that organizations deploying dedicated AI governance platforms are 3.4 times more likely to achieve high effectiveness compared to those that don’t.
23andMe’s customers left when they read what their data had been doing without them. Every AI agent running ungoverned and unaudited today is a draft of the same story just waiting to be written.
