For years, enterprise AI tools have primarily run on human-authored data, which has provided a stable frame for AI development, governance, and risk management.
But what happens when enterprise AI begins running on AI-authored data? What new risks emerge, how do we manage them, and what are the consequences of failing to do so? According to new research, we’re about to find out in real time.
The challenge is that many existing controls create a false sense of security. They assume AI models will operate only within the context of the users and systems that prompted them. In practice, though, model outputs often become summary documents, research, code, or other artifacts that are shared with broader groups and reused in future decisions. Even if the source data was security-trimmed, the resulting output can move far beyond its original permissions, becoming new context for the next judgment or iteration.
AvePoint’s 2026 State of AI Report, which surveyed 750 industry leaders, finds that more than a third of enterprise data is now generated by AI, with that figure projected to rise to 42% in the next 12 months. At the enterprise level, that means AI agents and assistants are increasingly making judgments based on self-generated information, potentially reproducing and amplifying incorrect assumptions, flawed outputs, and hallucinations.
This shift has profound implications for data protection and AI governance, and most organizations aren’t ready to respond.
Ungoverned AI Leads to Exponential Risk
When AI enters an enterprise environment, it takes the weaknesses already sitting in the data estate—stale records, misclassified files, over-shared documents, broad default permissions, unclear ownership—and scales them at an industrial pace. Governance flaws that a human team could once absorb or catch become structural liabilities the moment an autonomous system starts reading, reasoning over, and acting on that data thousands of times a day. And the foundation is shakier than most leaders assume: Gartner reports that 57% of organizations admit their data is not AI-ready.
That amplification is already visible. In AvePoint’s 2026 State of AI Report, nearly nine in ten organizations experienced an AI-related security incident in the past year. IBM’s Cost of a Data Breach Report points to why: 97% of organizations breached through AI lacked proper access controls, and 63% had no AI governance policy. These aren’t stories about clever attackers. In my experience, the most common exposure is accidental and non-malicious: the oversharing, broad permissions, and default access that widen the blast radius every time a new model or agent is switched on. Shadow AI widens it further. UpGuard found roughly eight in ten employees use unsanctioned tools, which IBM ties to an average $670,000 in added breach costs.
The proliferation of AI-authored data turns a manageable weakness into a compounding one.
When more than a third of enterprise content is machine-generated, systems increasingly make judgments based on their own prior outputs. Researchers at Oxford and Cambridge, writing in Nature, documented what happens next: models trained on recursively generated data suffer “model collapse,” steadily losing accuracy and diversity, the equivalent of photocopying a photocopy until the picture degrades.
Without governance, hallucinations and flawed assumptions get reproduced rather than corrected, and every ungoverned document becomes context for the next decision. Visibility, meanwhile, is moving the wrong way: AvePoint found the share of organizations that can’t tell whether employees are using unsanctioned AI tools nearly tripled in a year, from 6.3% to 17.6%, even as 84.1% now manage at least a petabyte of data and nearly 80% say more than half of theirs is over five years old.
Data Readiness Is Now AI Readiness
What organizations need is to stop treating this as a model problem and start treating it as a data problem. AI trust is built at the data layer, not inside the model. Gartner projects that through 2026 organizations will abandon 60% of AI projects unsupported by AI-ready data, and 63% of data leaders say they lack—or aren’t sure they have—the practices AI requires. Without a deliberate trust layer—an operational foundation of data protection, governance, and lifecycle visibility the organization owns itself—introducing agentic AI simply scales the confusion that already exists.
And that trust layer has to be operationalized, not documented once and filed away. It means privacy by design and by default, so protection is built into workflows rather than bolted on afterward. It means continuous visibility into what data AI can see, create, and act on, because point-in-time compliance says nothing about the configuration drift happening hour by hour (you can’t secure what you can’t see).
AI-generated data is here, and it isn’t slowing down. Whether it becomes an asset or a liability depends less on the sophistication of the models an organization buys than on the discipline of the data protection and governance beneath them.
The question every leader should ask is practical and straightforward: where does your organization still lack the visibility it needs to govern the data your AI can already touch? As AI-generated data continues to proliferate, it will become more important than ever to have workable, operationalized answers to questions like this one.
