Medical Imaging Contains More Than What Meets the Eye
A CT scan of a tumor contains far more information than anyone currently extracts from it. The image textures at the boundary between tumor and healthy tissue. The architecture of blood vessels feeding the mass. The density gradients that shift in ways no human eye can reliably detect. Standard practice reduces all of this to one measurement: the longest diameter, captured by Response Evaluation Criteria in Solid Tumors, a framework oncology has used for decades.
Radiology scans capture far more than tumor size, including information on tumor burden, heterogeneity and structure, and blood vessel network that sustains the tumor. The field of radiomics measures quantitative features from radiology images that describe patterns in texture, shape, and intensity that are not visible to the human eye. Advanced AI and statistical models can then utilize these features to predict patient outcomes like survival or treatment response.
As the AI oncology research community builds AI tools to further extract and interpret this information, it’s imperative to do so in a way that oncology therapy developers, clinicians, and ultimately patients, can trust.
Many early AI models were designed to maximize prediction accuracy without clear constraints on what they were learning. In one notable example, AI models trained to classify skin lesions were found to have learned that certain image markings correlated with malignancy, not because those features reflected cancer biology, but because they reflected clinical marks made to indicate suspicion. Advances in deep learning have taken this a step further. Today, many approaches rely on black-box AI – models that take in raw imaging scans and generate predictions directly, often learning from raw imaging data without relying on predefined features. While this has improved performance, it has also made it harder to understand what these models are actually learning.
As a result, some models may pick up signals tied to scanner differences or correlations within a patient cohort, in other words, statistical “shortcuts,” rather than true biology.
This does not make those models useless, far from it. Black-box approaches have driven real progress in AI across medicine and beyond. But in high-stakes settings like oncology, performance alone is not always enough.
Clinicians need to understand why a model is making a prediction, not just whether it is accurate.
Evolving Beyond the Black-Box: Glass-Box AI
Glass-box AI builds on radiomics and machine learning, but takes a more disciplined approach. Instead of searching broadly for any predictive pattern, models are designed around features that reflect known or observable aspects of disease biology, allowing them to be utilized as biomarkers.
In a glass-box model, you can answer three simple questions:
- What is the model measuring?
- Why does it matter biologically?
- How did it lead to this prediction?
That level of clarity allows AI to move from a research tool to something clinicians can rely on in practice.
What Glass-Box Design Looks Like in Practice
One example of an interpretable imaging biomarker approach focuses on quantifying tumor vascular architecture. Tumor-associated blood vessels are often disorganized, which can affect both drug delivery and immune cell infiltration. While these patterns are visible on routine imaging, they have historically been difficult to measure reproducibly.
Recent work has explored analyzing vessel properties such as twistedness, branching, and volume from routine CT scans, and combining them into a single biomarker, the Quantitative Vessel Tortuosity (QVT) score. In retrospective, multi-institutional studies, this score has shown associations with outcomes such as survival following immunotherapy.
Instead of training a model to simply predict outcomes, QVT was built by first identifying measurable vascular patterns, such as vessel twistedness, branching, and shape. In a multi-institutional study of 682 patients and over 1,300 CT scans, this biomarker predicted patient survival after immunotherapy treatment, including in patients where standard markers like PD-L1 were less informative. Importantly, the model’s predictions can be traced back to specific biological properties. In other words, the model can show its work.
What Glass-box AI Means for Clinical Development
This kind of transparency matters in drug development. When a biomarker is easy to interpret, it is easier to test, validate, and explain to regulators. It also becomes more useful when things do not go as planned.
If a clinical trial fails, an interpretable model can help answer an important question: did the drug fail, or did we select the wrong patients? That insight can save time, cost, and future effort.
As expectations from regulators continue to rise, models that can explain their predictions may offer an advantage. One lesson is clear: the challenge in medical AI is not performance, but building models that reflect disease biology. Those that do are more robust, more generalizable, and easier to trust. This is the foundation of glass-box AI — aligning models with the scientific reasoning that medicine depends on.
A Better Way Forward
Glass-box AI is not just a response to the limits of earlier approaches. It points to a better way forward. When AI is used to guide clinical decisions, it should be able to explain itself clearly. Not just to meet regulatory standards, but to meet the expectations of good science and patient care.
As these tools continue to develop several challenges remain. Imaging-derived biomarkers must be standardized across scanners, institutions, and patient populations. Reproducibility and external validation are essential before clinical adoption. In addition, integrating these tools into clinical workflows requires alignment with regulatory and operational frameworks.
Medical imaging has always contained more information than we use today. We now have the tools to extract it in a way that is interpretable and biologically grounded.
That is what will turn AI from a promising technology into a trusted part of medicine.
About the Author
Anant Madabhushi, PhD, is Chief Scientific Officer at Picture Health and the Executive Director of the Emory Empathetic AI for Health Institute. He has published more than 500 peer-reviewed papers on computational imaging, radiomics, and machine learning in oncology.
Disclosure: The author serves as Chief Science Officer at PictureHealth, which developed the QVT biomarker described in this article.
