Generative and agentic AI are forcing organisations to rethink many of the assumptions underpinning traditional model risk management. Joe Breeden, President of the Model Risk Managers International Association, argues that model risk management should no longer be viewed simply as a regulatory requirement, but as a critical mechanism for protecting the value generated by AI investments. As AI capabilities become more autonomous and complex, financial institutions must adapt governance, validation and monitoring approaches to manage emerging risks while maintaining accountability, resilience and trust across the organisation.
CeFPro has begun a long-term strategic partnership with the Model Risk Managers International Association (MRMIA), bringing leading practitioner insight on model risk management to Connect Magazine, our events, and the wider risk management community. As part of the alliance, MRMIA will contribute a new series of expert articles examining the challenges reshaping the discipline. In the first installment, MRMIA President and Deep Future Analytics Founder and CEO Joe Breeden explores why generative and agentic AI demand a fundamental rethink of traditional model risk management.
Financial institutions must stop treating model risk management as a regulatory cost and start recognizing it as the mechanism that protects returns from artificial intelligence.
As generative and agentic AI move deeper into business processes, effective risk management will determine whether promised productivity gains become sustainable value or introduce financial, regulatory, and reputational exposures.
Joe Breeden argues that AI provides an opportunity to reset the model risk conversation after years in which MRM has often been regarded as a compliance or audit burden.
“The risk management is how you guarantee the return on investment,” he said, calling for projected AI returns to be risk-adjusted rather than presented independently of the risks required to achieve them.
The urgency is partly driven by employee behavior. Breeden cites research involving 48,000 employees that indicated 44% had knowingly, or to some extent knowingly, used AI in ways that contravened organizational policies or guidelines.
He rejected the suggestion that this implied widespread malicious intent, arguing instead that employees frequently use AI because they are simply trying to complete work efficiently.
That creates the possibility of well-intentioned breaches, including employees entering company information into public systems or allowing AI to influence decisions beyond permitted boundaries.
The problem becomes more complex when AI itself creatively interprets policy. Breeden described how an AI assistant might correctly tell an employee that a minimum-balance requirement cannot be waived but then suggest changing the customer’s account type so the requirement would not apply until later.
Such behavior, he says, could “transform your AI assistant” into “an AI accomplice for circumventing policy in an instant,” despite neither the employee nor the system explicitly setting out to break those rules.
Traditional MRM principles remain relevant. Model purpose, documentation, independent validation, governance, accountability, ongoing monitoring and inventories still matter.
But Breeden argues that many assumptions that have often made traditional model governance workable actually break down when applied to generative AI.
Conventional models are comparatively deterministic, operate within understood boundaries and can be assessed using established
performance metrics.
Large language models can produce different outputs from identical inputs, have an effectively enormous universe of potential inputs and outputs, and remain difficult to explain.
Generative AI therefore needs to be distinguished from conventional
machine learning. Breeden argued that nonlinear machine-learning models can largely be accommodated through adaptations to established MRM, while generative AI demands a fundamentally different perspective.
That includes reconsidering what constitutes a model. If every technology incorporating AI is classified as one, institutions could effectively find their entire technology estate entering the model inventory.
Breeden instead proposed applying model risk management wherever model risk exists.
The same chatbot might simply retrieve information in one use case while contributing to a lending decision in another. Its classification should consequently reflect what it does and the risks created by its use rather than its underlying technology alone.
That risk-based approach becomes increasingly important with agentic AI capable of altering databases or writing directly to systems of record without human intervention. Even AI classified as a tool may therefore require controls and monitoring.