EXECUTIVE SUMMARY
AI mediates reality, not just decisions. AI doesn’t just change how decisions are made. It changes what those decisions are made from and how that information is constructed.
If organizational structures define who is accountable for decisions, data and information flows determine the quality and integrity of those decisions.
Organizations are now operating within a dual-risk environment. First, decisions are increasingly based on data that may be biased, incomplete, outdated, or misaligned with reality. Second, AI systems are acting as intermediaries that aggregate, filter, summarize, and synthesize information before it reaches decision-makers.
This creates a structural shift.
Key Insights
-
Data quality directly shapes decision quality in AI-enabled environments
-
AI systems increasingly influence how information is constructed, not just delivered
-
Bias and gaps in data can be amplified through aggregation and summarization
-
Lack of transparency in AI-generated outputs weakens human judgment and accountability
-
Data governance and decision governance are now inseparable
INTRODUCTION
If the first challenge in AI governance is defining who is accountable for decisions, the next challenge is understanding what those decisions are based on and how that information is constructed.
Clear decision authority alone doesn’t ensure sound outcomes. Even well-governed organizations can make poor decisions when the underlying data is flawed or when information is filtered through opaque AI systems.
AI systems are no longer functioning solely as analytical tools. They are increasingly becoming the interface through which information is accessed, interpreted, and acted upon.
This creates a new layer of organizational risk. Decision-makers aren’t only dependent on the quality of the underlying data. They’re increasingly dependent on how AI systems select, combine, prioritize, and present that information.
The result is a growing gap between underlying reality and what decision-makers perceive as reality. In traditional environments, professionals often reviewed source material directly, evaluated competing perspectives, and applied contextual judgment. In AI-enabled environments, that process is increasingly mediated by systems that summarize, synthesize, and interpret information before humans engage with it.
This changes the nature of decision-making itself. The risk is no longer only whether a model produces an incorrect output. The risk is that organizations may gradually lose visibility into how information is constructed, what assumptions shape it, and where critical context may be missing.
THE THREE LAYERS OF DATA RISK
AI-related data risk doesn’t operate at a single level. It exists across three interconnected layers that together shape how organizations perceive and act on information.
Foundational Data Risk: Distorted Input
The first layer involves the quality, completeness, and representativeness of underlying data. AI systems rely on historical and structured datasets to generate outputs. When those datasets are biased, incomplete, outdated, or poorly governed, the resulting outputs reflect those same limitations.
Zillow Offers illustrates this challenge. Zillow used an AI-driven pricing model to power its iBuying program, automatically purchasing homes it believed were undervalued. As housing markets became increasingly volatile, the models struggled to accurately forecast home prices at scale, contributing to significant losses and the eventual shutdown of Zillow Offers in 2021. Leadership later cited forecasting limitations as a key factor in the program’s collapse.,
The issue was not simply the algorithm itself. The failure reflected a broader assumption that historical housing data could reliably predict dynamic and volatile market behavior. This illustrates a foundational governance challenge: AI systems inherit the limitations of the data environments on which they depend.
Contextual and Representational Risk: Distorted Applicability
The second layer emerges when data doesn’t accurately represent the real-world context in which decisions are being made. Even when data is technically accurate, it may still omit critical variables, oversimplify complex conditions, or be applied outside the environment for which it was originally designed.
McDonald’s AI-powered drive-thru pilot demonstrates this challenge. Beginning in 2021, McDonald’s partnered with IBM to pilot AI-powered drive-thru ordering. Real-world conditions, including accents, background noise, and inconsistent audio quality, created persistent accuracy challenges. After three years of testing, McDonald’s ended the program in 2024 and returned ordering responsibilities to human staff. ,
The challenge wasn’t simply bias in the traditional sense. It was the mismatch between generalized data models and highly specific human realities. In high-stakes environments, context matters as much as accuracy.
Aggregation and Intermediation Risk: When AI Shapes What We See
The third and increasingly important layer involves AI systems acting as intermediaries between users and information. Unlike foundational data risk, which affects the quality of inputs, or contextual risk, which affects how information applies to real-world situations, intermediation risk emerges when AI shapes what information is surfaced, how it is organized, and what decision-makers ultimately see.
Recent reporting on Amazon and Walmart highlighted employee concerns regarding the growing role of AI in workplace management and HR-related processes. According to worker surveys and interviews, employees described AI systems increasingly influencing scheduling, productivity measurement, accommodations workflows, issue escalation, and other workforce-management activities.
While managers and HR professionals may retain formal decision authority, workers reported that information is often categorized, prioritized, filtered, or surfaced through automated systems before human review occurs.
The governance challenge extends beyond whether AI makes the final decision. As AI increasingly shapes what information decision-makers see, how issues are framed, and which cases receive attention, it influences the informational environment in which human judgment operates.
AI systems increasingly aggregate, synthesize, summarize, and prioritize information before it reaches decision-makers. In the process, they influence:
-
What sources are included
-
Which perspectives are emphasized
-
How information is weighted
-
What context is omitted
-
How uncertainty is presented
Users rarely see these decisions directly. This creates a structural risk. AI-generated outputs can appear neutral, authoritative, and complete while being shaped by incomplete, biased, or uneven source distributions.
The issue isn’t only that AI systems can be wrong. It’s that they can present a constructed version of reality without transparency into how that version was formed. In areas such as financial analysis, geopolitical events, hiring, healthcare, and public policy, this can materially influence perception and decision-making at scale.
WHEN THESE RISKS COMPOUND
Individually, each layer creates governance challenges. Together, they can fundamentally alter how organizations perceive and respond to reality.
These layers don’t operate independently. They reinforce each other. Poor foundational data feeds into models. Contextual gaps distort how information is interpreted and applied. Aggregation systems reshape how outputs are ultimately presented to decision-makers, often with limited visibility into the assumptions, prioritization mechanisms, and source-selection criteria behind them.
By the time information reaches a human decision-maker, it may be several layers removed from its original source.
At each stage, assumptions are introduced, transparency decreases, and the distance between underlying reality and perceived reality can widen.
This is where risk begins to scale beyond isolated technical errors and becomes an enterprise governance issue. Organizations may believe they’re making informed decisions while operating from incomplete or distorted informational environments.
THE GROWING TRANSPARENCY GAP
The interaction of these layers creates a growing transparency gap inside organizations.
Decision-makers are still expected to remain accountable for outcomes, but they often lack visibility into:
-
Where data originated
-
How information was selected or filtered
-
What assumptions were introduced during synthesis
-
Which perspectives may have been excluded
-
How outputs were prioritized or framed
Without this visibility and context, effective judgment becomes increasingly difficult. Accountability remains, but control diminishes.
This creates a dangerous dynamic in AI-enabled environments. Organizations may formally preserve human oversight while functionally reducing the ability of humans to independently evaluate what they’re seeing.
Over time, professionals may begin relying on AI-generated summaries and interpretations as substitutes for deeper analysis, especially under conditions of speed, scale, and operational pressure. The result isn’t only automation risk. It’s cognitive dependency risk.
DATA GOVERNANCE IS NOW DECISION GOVERNANCE
Organizations have traditionally treated data governance and decision governance as separate disciplines. In AI-enabled environments, that separation no longer holds. Data determines the inputs to decisions. AI systems shape how those inputs are interpreted. Humans act on the outputs. For CIOs and technology leaders, this means governance responsibilities now extend beyond system performance to the integrity of the information ecosystem supporting enterprise decisions. If any part of that chain lacks integrity, transparency, or accountability, governance failure can emerge even when technical systems appear to function properly.
Effective AI governance must therefore extend beyond model validation and compliance controls. It must also address:
-
Data quality and representativeness
-
Contextual relevance to decision environments
-
Transparency in AI-generated outputs
-
Visibility into aggregation including the sources of information and synthesis processes
-
Alignment between information flows and accountability structures
In practice, organizations should evaluate AI-mediated decision environments through four governance lenses: data integrity, contextual relevance, transparency of information flows, and meaningful human oversight.
This means establishing accountability for data quality, documenting source provenance where feasible, validating AI-generated outputs in high-impact use cases, and preserving human review for decisions involving significant operational, financial, regulatory, or reputational consequences.
Organizations that fail to govern these layers risk making decisions based on incomplete, distorted, or unverifiable representations of reality.
GOVERNANCE ACTIONS
The objective is not to eliminate AI mediation, but to ensure decision-makers can understand, challenge, and validate the information shaping critical decisions. Organizations can strengthen governance of AI-mediated information environments by incorporating the following questions into decision-making processes:
-
What am I seeing?
Is this raw information or an AI-constructed interpretation?
-
Where did it come from?
Do I understand the data’s origin, completeness, and limitations and any potential biases?
-
Who shaped it?
Which systems, models, or intermediaries filtered, summarized, or weighted the information?
-
Why should I trust it?
What assumptions, omissions, or contextual gaps sit underneath the output?
-
When must human judgment re-enter?
Where is independent review essential to avoid blind spots or over-reliance?
THE BOTTOM LINE
In AI-enabled environments, organizations are not only making decisions with data, they’re increasingly operating within constructed representations of reality generated by AI systems.
AI doesn’t just amplify decision-making. It amplifies the data and information structures behind those decisions.
Organizations that fail to address data quality, contextual integrity, and transparency aren’t just exposed to technical risk. They’re making decisions based on constructed and potentially distorted representations of reality. This isn’t separate from governance. It’s a continuation of it.
Accountability must extend beyond decisions themselves to the data, information flows, and AI-generated interpretations those decisions depend upon.
Without that, accountability exists in name only.
ABOUT THE AUTHORS
Anastasia Raissis is a governance, risk, and responsible AI strategist, former Amazon and AWS executive, founder and CEO of The Achillia Group and Head of Strategy at AI 2030, focused on AI accountability and regulatory risk.
Howard Waterman is a communications and crisis strategist and advisor, former executive at Verizon and Moody’s where he led communications teams supporting multi-billion-dollar business units, founder of The Waterman Group consultancy and member of the AI 2030 board of advisors.
Howard Waterman
