● Why defensibility breaks when obligation, policy, control, alert, and outcome cannot be traced end-to-end

● What a shared data and policy layer changes across trade surveillance, communications surveillance, archiving, and policy management

● Why consistent, cross-framework control logic is essential to credible oversight

● What “explainability” needs to look like for investigators and validators

● How to operationalize scenario-based testing and residual risk governance

Following Advanced Model Risk USA, we sat down with Caitlin Like, Global Head of Account Management and Customer Success at Behavox, to refl ect on the conversations that resonated most with model risk and compliance leaders. She shares why fragmented surveillance and governance models are creating defensibility gaps for fi nancial institutions,  and how a unifi ed controls framework can close them. She also shares what it takes to build an AI-driven control environment that survives model validation and regulatory review.

When regulators examine a firm's surveillance and controls program today, what is the most common gap they find, and why does it persist?

 The most consistent gap isn't a missing tool or an underperforming model. It's the inability to produce a coherent, end-to-end evidence trail that connects a regulatory obligation to a specific control, through to detection, investigation, and corrective action. Regulators increasingly want to trace that full story — and many firms simply cannot tell it.

The reason it persists is structural. In a typical financial institution, policy and obligations management sits in one system; preventive controls like information barriers and pre-clearance live in another; and detective controls, including communications surveillance, trade surveillance, and transaction monitoring, run on separate platforms with independent alert pipelines. Every one of those functions may individually perform well. But when they operate in isolation, you lose the connective tissue. You cannot reconstruct a case without weeks of manual assembly across multiple teams. You cannot demonstrate that what you detected was actually tied to what you were obligated to monitor.

Fragmentation is a recurring operational pattern in governance-related findings across supervisory cycles — reflected in FCA, ECB, and SEC examination themes over recent years — and it persists in part because it is structurally embedded in how most institutions have built their control functions over time. Regulators do not formally prescribe specific technologies, but through examination, SREP dialogue, and model risk review, supervisors increasingly form views on whether a firm's architectural choices support genuinely effective controls — and fragmented architectures have featured in findings across multiple jurisdictions.

You advocate for replacing fragmented models with a unified controls framework. What does unification actually mean in practice - and what does it not mean?

Unification is a control-system property, not a technology property. It does not necessarily mean collapsing everything onto a single platform, though that can help. What it does mean is that every regulatory obligation is mapped to specific preventive and detective controls through a shared, version-controlled taxonomy; that any case can be reconstructed end-to-end from system records alone; and that the program can demonstrate consistent control coverage across channels, populations, and languages — with documented model performance benchmarks, residual risk assessments, and validation conducted independently of model development — not just for flagship scenarios.

In practice, unification requires four things: a shared taxonomy so that different functions are classifying the same risks the same way; automated data linkage between preventive and detective workflows; defined handoff protocols so that context is not lost as an alert moves from detection to investigation to remediation; and a governance structure that explicitly owns the seams between those functions.

What it does not mean is a single monolithic system that all functions must conform to. The critical asset is the shared data and policy layer — the common thread that connects obligation to outcome. Whether that is achieved through one platform or through well-integrated systems is a technology decision. The operating model requirement is the same either way.

It is also worth being precise about scope. The framework described here operates within the surveillance and conduct risk perimeter. Institutions should separately assess how their integrated control architecture aligns with jurisdiction-specific obligations — including DORA ICT risk management requirements for EU-regulated fi rms, SM&CR accountability mapping under the UK regime, and applicable model risk governance standards — as supervisory expectations across these frameworks continue to evolve.

How does bringing trade surveillance, communications surveillance, archiving, and policy management onto a common layer change the investigator experience — and ultimately, the quality of outcomes?

Significantly. When an alert fires in a siloed environment, the investigator typically sees the alert and very little else. To understand whether a fl agged communication is genuinely concerning, they may need to know whether a trade occurred around the same time, whether there was a barrier crossing, whether the individual was on a restriction list, what the archived record shows — and assembling that picture manually takes time, introduces inconsistency, and increases the likelihood of a wrong disposition.

When trade surveillance, communications surveillance, archiving, and policy management share a common data and policy layer, the investigator sees all of that context automatically, in a single workflow. The quality of decisions improves. The speed improves. And critically, the evidence chain is complete and auditable from the outset — you do not have to reconstruct it later.

There is also an important feedback dimension. When you can see across the full control environment, you can identify patterns: cases where detective controls are repeatedly finding the same conduct category, suggesting that a preventive control needs adjustment. That closed loop — from detection through to prevention — is the architecture through which institutions can pursue sustained improvement in control effectiveness, rather than managing the same recurring exposure reactively. Whether that translates into measurable risk reduction depends on operating model execution, governance maturity, and ongoing independent validation.

One of the themes you emphasized in your session is consistent, cross-framework control logic. Can you explain why consistency matters so much, and what happens when it breaks down?

Control logic refers to the rules, thresholds, and detection models that determine what gets flagged and how. When that logic is inconsistent across frameworks — when the same underlying risk is monitored differently in trade surveillance versus communications surveillance, or when different regulatory frameworks are implemented with different coverage assumptions — you create blind spots and you undermine your own defensibility.

The most obvious consequence of inconsistency is gaps. If your communications surveillance covers a risk scenario that your trade surveillance does not, or vice versa, a sophisticated actor can exploit the seam. But there is a subtler consequence that matters just as much: inconsistency makes it very hard to demonstrate to a regulator that your control system is coherent. If the same obligation is implemented differently by different teams, the evidence you produce in response to an examination is non-comparable, and your account of the control environment is difficult to defend.

Consistent, cross-framework control logic does not mean identical logic everywhere — different asset classes and communication channels genuinely require different approaches. It means shared classification, common coverage principles, and governance that reviews the logic holistically rather than function by function. That is how you move from a collection of controls to a control system.

AI is now central to communications surveillance. What does meaningful model explainability look like in this context, and why does it matter beyond satisfying a validation requirement?

Explainability in AI-driven surveillance means being able to articulate, in plain terms, why a model flagged a specific communication — what it understood about that message, in context, that suggested potential misconduct. It also means being able to demonstrate how the model was trained, on what data, with what coverage across languages and risk typologies, and how its performance has been validated against independently verified test cases, including known true-positive expressions drawn from historical cases or supervisory typologies.

The reason it matters beyond validation is that explainability is the mechanism through which investigators can trust and act on model outputs. If a reviewer cannot understand why an alert was generated, they cannot make a well-reasoned disposition decision. Over time, that erodes review quality and increases the risk of both under-escalation and over-escalation.

From a model risk perspective — and this is directly relevant to the SR 11-7 and OCC 2011-12 framework in the US, and SS1/23 in the UK — one of the most practical things institutions can do is establish a structured testing framework that introduces those verified test cases into monitored data streams and measures detection rates against that ground truth. Institutions that can present quantified, scenario-based testing evidence — documenting what their models detect, what they miss, and what residual risk has been accepted and on what basis — are generally better positioned to answer examiner questions than those relying solely on qualitative capability narratives. This reflects the direction of travel in model risk supervisory guidance across the EU, UK, and US.

You referenced residual risk — the exposure that remains after primary AI controls are applied. How should institutions think about managing it, and what does best practice look like today?

Residual risk is an unavoidable feature of any AI-based detection program. Even a very high-performing model will miss institution-specific phrasing, highly contextual expressions of misconduct, or language patterns that did not appear in the training corpus. The question is not whether residual risk exists — it does — but whether you have a governed, documented approach to managing it.

The defensible approach is to treat detection as layered, using AI with complementary techniques to extend coverage where needed, and to govern each component with clear scope, independent validation, and documented residual risk. Institutions should supplement vendor-provided testing with their own independent validation in line with applicable model risk management requirements.

What makes any such framework defensible is not just the architecture — it is the testing. Institutions should be running regular scenario-based testing that quantifies detection rates and residual risk as a percentage of verified test cases. That gives you an evidence base for the conversation with model risk and with regulators: here is what we monitor for, here is how we tested it, here is what our controls catch, and here is the residual exposure we have accepted and why.

We would characterise this layered approach as one defensible architecture — institutions should evaluate it against supervisory expectations in their specific jurisdiction.

Without that evidence base, firms are left with a narrative. With it, they have a control environment they can defend.

Looking ahead, what do you see as the most underestimated challenge for institutions trying to build a control environment that is genuinely future-proof?

The talent and operating model question is consistently underestimated. Institutions invest heavily in technology — and the technology landscape has genuinely transformed what is possible — but integration is a cross-functional discipline that requires skills spanning compliance, data governance, technology, and model risk. Those profiles are rare, and programs that depend on a small number of cross-domain individuals face key-person risk and cannot scale.

The other thing I would flag is the pace of change in communication channels and conduct typologies. Firms that have built their control environments around the channels and risk scenarios that existed three years ago need to be thinking now about how they extend coverage to emerging channels, new languages, and automated or machine-generated communications — an area where the regulatory framework is still developing and where firms should assess their obligations under emerging AI-specific guidance in applicable jurisdictions. Programs that are likely to be better positioned for future supervisory scrutiny are those that can demonstrate not only current control coverage but also a documented, governed roadmap for extending that coverage to emerging channels and typologies — acknowledging that what satisfies any specific examination remains a supervisory determination.

The underlying principle is the same as it has always been: effective systems and controls require ongoing investment, continuous testing, and a governance structure with clear accountability for the end-to-end loop. Technology makes that possible at scale. But the operating model has to support it.

Caitlin Like is Global Head of Account Management and Customer Success at Behavox, where she leads strategic customer success programs that strengthen client outcomes across the Behavox platform. She recently spoke at Advanced Model Risk USA on ‘The Unified Controls Framework,’ sharing practical guidance for building regulator-ready evidence chains across modern control environments. 

If you are evaluating how to connect obligations, policies, surveillance, and evidence into a regulator-ready control story, speak with Behavox about a working session on unified controls and evidentiary readiness. 

The views expressed are those of the individual speaker and are provided for informational purposes only.