Skip to content

Content Library

Back to Articles

Article July 8, 2026

The Failure Syndrome – How Models Fail When Data Fails

The Failure Syndrome – How Models Fail When Data Fails

Financial institutions must rethink how they build and govern valuation models as data becomes scarcer, markets more volatile, and AI more capable. A senior treasury executive argued that artificial intelligence should strengthen human judgment rather tha

  • Poor quality and limited market data remain the greatest challenge for valuation models
  • AI can improve model performance through better data analysis and synthetic data generation
  • Greater predictive accuracy must be balanced against explainability and governance
  • Human judgment should remain central to valuation decisions and model overrides
  • Strong documentation and evidence are essential when challenging model outputs
  • Continuous governance helps maintain confidence in valuation frameworks as markets evolve

Artificial intelligence may offer financial institutions powerful new ways to improve valuation models, but it will never remove the need for human judgment, governance and accountability, according to a senior treasury executive speaking at a recent financial services conference.

As banks contend with increasingly volatile markets, illiquid assets and imperfect data, the speaker argued that organizations should focus less on finding perfect models and more on understanding the limitations of the information that feeds them.

"Garbage in is garbage out," he said, describing poor quality data as the fundamental challenge behind model performance. "Data is not good. And that is going to affect the quality of your model."

Rather than viewing model risk solely through the lens of mathematical techniques, the presentation emphasized that valuation challenges originate much earlier in the modeling process.

Illiquid markets, sparse observations and inconsistent data definitions all contribute to uncertainty long before a model begins generating prices.

The senior treasury executive illustrated the issue using the valuation of illiquid bank loans. Unlike exchange-traded instruments, these assets often have little or no observable market data, making traditional valuation approaches difficult to justify.

During periods of market disruption, even products that normally trade actively can quickly become illiquid, removing the very observations on which many valuation frameworks depend.

"If you don't have data," he said, "it's very difficult to come up with the right equation, let alone calibrate your equation and come up with some parameters."

The consequences extend well beyond technical model performance. Incorrect valuations can affect profit and loss, trigger disputes with front-office teams and create difficult conversations with auditors and supervisors who expect firms to demonstrate that their pricing remains appropriate despite limited market evidence.

 

Throughout the session, the speaker stressed that valuation is not confined to trading activities.

Fair value calculations influence multiple areas of the balance sheet, from accounting disclosures and banking book positions to credit spread risk and model risk adjustments.

For many institutions, the overwhelming majority of assets remain effectively illiquid, creating ongoing pressure to develop robust valuation methodologies despite limited data availability.

The presentation highlighted several practical challenges. Available market observations are often concentrated in only a small number of products or borrower types, leaving large portions of portfolios represented by only a handful of comparable transactions.

As additional risk characteristics such as maturity, collateral, geography or sector are incorporated, available observations become even thinner.

"Your data here is skewed," the executive explained. "You've got a number of buckets with a lot of observations, but in most buckets you have not so many observations."

Artificial intelligence, he argued, can play a valuable supporting role throughout this process.

Machine learning can automate data extraction, classify loan characteristics, identify the variables that matter most and even generate synthetic observations that improve statistical coverage in sparsely populated data sets.

The speaker described internal experiments where machine learning techniques improved model fit while reducing statistical uncertainty by generating realistic additional observations that complemented existing market data.

However, he cautioned that better predictive performance alone is insufficient to justify deployment.

"We know that the AI model is doing a better job," he said. "But it adds another layer of unexplained complexity."

That complexity presents its own governance challenge. Validation teams, auditors and regulators must understand not only the outputs produced by AI but also the reasoning behind them. Models that cannot be adequately explained may prove difficult to defend even if they produce more accurate results.

Events you may be interested in

View all of our upcoming events

Sign Up to Our Newsletter

Sign up

Connect Plus

Unlock more with Connect Plus membership

This content is available to Connect Plus members.