Skip to content

Content Library

Back to Articles

Article May 22, 2025

Gen AI - The Model Risk Dilemma No One’s Solved … Yet

Gen AI - The Model Risk Dilemma No One’s Solved … Yet

Andreas Simou, Managing Director, Center for Financial Professionals

As financial firms race to integrate GenAI into model documentation, research and analysis highlights the complex, evolving art of validating AI with AI.
  • ValidMind presented a GenAI use case for automating test result interpretations in model risk
  • Interpretations combine general test explanations with specific result analysis
  • Subjectivity in technical narratives makes consistency and compliance difficult
  • Metric-based tools like RAGAS offer scalable evaluation but rely on models to judge models
  • Historical benchmarking and human-in-the-loop feedback improve quality
  • Modular testing, like numeric consistency checks, reduces hallucinations
  • Validation now requires interdisciplinary collaboration across legal, tech and compliance teams
  • GenAI updates demand agile inventory management and adaptive workflows
  • Transparency and traceability are critical for regulatory acceptance
  • Best practices blend automation with human oversight for scale and accuracy

display-middle-ads-here

Despite being the number one topic occupying the headspace of risk managers all over the world, the thorny issue of generative artificial intelligence (GenAI) is one that – for now at least – seems still to elude consensus.

The practical realities and emerging strategies for implementing and validating GenAI were certainly front of mind at CeFPro’s USA conference  in the context of model risk management (MRM).

The session focused on a use case from ValidMind that shed light on the challenges and innovations involved in integrating GenAI features into tools used for automating model documentation and validation.

The core of the presentation centered around a GenAI-driven feature designed to automate the interpretation of test results, a function critical to both developers and validators operating within tightly regulated industries.

Though specific in its nature, the use case also served as a microcosm for broader questions confronting the financial sector.

How should organizations validate AI-generated content? What constitutes a ‘well-written’ interpretation of technical model results? And how can subjective judgments be systematized to ensure consistency and regulatory compliance?

In this case, the GenAI feature in question was used to interpret outputs from various tests – from correlation matrices to scatter plots – and generate narrative explanations that assessed test methodologies and gave context for specific results.

The speaker – a senior figure within the risk management community – emphasized the dual structure of such interpretations as both a general explanation of the test and a specific analysis of the results.

The challenge, it emerged, lies in the subjective nature of such interpretations, where domain experts may disagree on what aspects are most relevant and how much detail is appropriate.

The evaluation of these interpretations presents its own array of difficulties.

While expert-labeled benchmarks represent the gold standard, they are inherently limited by human subjectivity and resource intensity.

As an alternative, metric-based evaluations using open-source tools like Ragas provide scalable ways to assess qualities such as faithfulness, relevance, and logical consistency.

That approach, though, is not without irony, given it relies on models to judge models, which raises philosophical and practical concerns around circular validation.

Another method explored was benchmarking against historical reports that exemplify high standards, but this too falls short of universality, especially as organizational preferences and regulatory expectations evolve.

The most effective approach, it was argued, involves integrating human feedback into the loop.

Through iterative testing and development, several best practices emerged.

A principle common to all of them was a multifaceted approach to evaluation that blends automation for scalability with targeted human oversight to ensure depth and accuracy.

Events you may be interested in

View all of our upcoming events

Sign Up to Our Newsletter

Sign up

Connect Plus

Unlock more with Connect Plus membership

This content is available to Connect Plus members.