Jessica Radow explores what financial institutions can learn from robotics as AI moves from generating information to taking action. As agentic AI begins to make decisions and execute tasks, governance must expand beyond model accuracy to address authority, accountability, human intervention and ongoing oversight.
Over the past several years, I’ve had the opportunity to work with a number
of Physical AI and robotics companies as they’ve brought autonomous systems
into real-world environments.
One thing has become increasingly clear: the hardest customer conversations
rarely center on the AI itself. They center on trust.
The Rise of AI That Acts
As long as AI is generating information, governance is largely about the
model: how it is trained, validated, monitored and explained.
Those conversations remain important as financial institutions deploy AI
across lending, fraud detection, trading, compliance and customer service.
Agentic AI changes that equation. As AI begins performing work rather than
simply generating information, organizations are no longer asking only whether
the AI is accurate.
They are asking what decisions it can make, when humans should intervene,
and how it will operate safely within business processes.
The questions shift from model performance to authority, accountability
and oversight.
What authority has AI been given? Where are the boundaries? Who is
monitoring behavior after deployment? What happens when something unexpected
occurs? When should a human intervene? Who ultimately remains accountable?
Physical AI has answered these questions for years. Autonomous mobile
robots do more than navigate from point A to point B.
They interact with people, equipment and workflows while operating within
established business processes.
The challenge is not simply whether the robot completes its task, but how
it behaves when conditions change, when humans should intervene and who owns
those decisions.
No organization would deploy autonomous systems without addressing these
challenges first.
Governance is built into these systems from the outset because failure
can disrupt operations, create safety risks, damage equipment and undermine
trust.
These disciplines give customers confidence to adopt autonomous systems
in real-world environments.
Why Financial Institutions Should Care
Banks may not be deploying robots in warehouses, but they are beginning to
deploy AI systems that can make decisions and take action.
Bank of England Deputy Governor Sarah Breeden recently highlighted the
emergence of agentic AI in payments and trading, noting that the financial
system is evolving from AI that generates content to AI that autonomously
carries out sequences of actions.
She also warned that existing governance frameworks were not designed
with autonomous agents in mind.
In treasury operations, AI is moving beyond forecasting cash positions and
detecting anomalies to coordinating liquidity, initiating payment instructions
and supporting transaction execution.
As these systems become more autonomous, hallucinations, objective drift
and unauthorized actions become operational rather than model risks.
Lessons from Physical AI
Financial institutions do not need to start from scratch. Physical AI
offers practical lessons beyond robotics.
Every autonomous system needs clearly defined authority. Organizations
should establish clear guardrails around what AI can do independently, where
the limits are, and when decisions should escalate to humans before systems
begin
acting autonomously.
Validation is only the starting point. Physical AI systems are
continuously monitored because environments change, unexpected situations arise
and performance evolves over time.
Agentic AI requires the same continuous governance to monitor performance,
detect unexpected behavior and enable timely intervention.
Successful autonomous systems are designed for uncertainty. Rather than
assuming everything will go according to plan, they are built to fail safely,
recover gracefully and escalate when appropriate. Financial institutions should
adopt the same mindset as agentic AI becomes embedded in business processes.
Key Takeaways
As AI systems become more autonomous, governance must extend beyond
evaluating model performance.
Leaders should ask what authority is delegated to AI, which decisions should
always remain with humans, whether the organization can explain, audit, and
defend AI actions after deployment, whether there is visibility into production
behavior as systems interact with people and other technologies, whether teams
can intervene quickly when conditions change, and who owns ongoing accountability
throughout the lifecycle.