Craig Spielmann revisits earlier concerns regarding artificial intelligence and argues that recent developments have moved many of those risks from theory to reality. Citing incidents involving autonomous AI systems accessing external environments and displaying unintended behaviour, he highlights how increasingly capable models are challenging conventional governance assumptions. The article explores cybersecurity implications, including AI-assisted vulnerability discovery and autonomous attack capabilities, alongside concerns raised by central banks and international regulators. Spielmann argues that financial institutions should pay close attention to the speed at which AI capabilities are advancing relative to governance, oversight and risk-management frameworks.
Earlier this year, in an article I wrote for Connect Magazine
titled ‘AI: King, Benefactor ... or Despot’, I warned that artificial
intelligence could ultimately outrun humanity’s ability to control it.
Let’s be honest - the argument could easily have been dismissed
(and probably was, by some) as a dramatic extrapolation of where the technology
might eventually lead.
My central contention was stark: AI is becoming more than a
productivity tool, and society may struggle to control systems that become more
capable than the humans supervising them.
I made no apology for that then, and recent events, I would
argue, have made that warning considerably harder to treat as science fiction.
The clearest evidence comes from cybersecurity.
In May 2026, Google’s Gemini autonomously breached systems
belonging to three companies during testing by independent evaluator Irregular.
According to Reuters, the model used publicly available
information to obtain or guess credentials and access systems it believed were
within the authorized scope of the exercise. Google confirmed the incidents and
said the organizations were notified.
Anthropic has
encountered similar problems. In September, it disclosed that a prototype
Claude model had accessed external systems without authorization during
testing.
The incident was
particularly significant because it had gone undetected for months despite an
earlier review. Other Anthropic models had previously breached internal company
systems during evaluations.
To be clear, none
of this demonstrates that AI has independently decided to turn against
humanity.
It does, however,
expose something more immediately relevant to financial institutions:
increasingly autonomous systems can behave in ways their developers did not
intend or anticipate.
That matters
because their capabilities are advancing rapidly.
The Bank of
England reported in July that frontier models can conduct multi-stage attacks
against vulnerable systems and identify and exploit weaknesses with little
human input. In one controlled reverse- engineering exercise, a frontier model
completed in 10 minutes and 22 seconds a task estimated to take a human expert
around 12 hours.
The financial
consequences could be profound. The Bank warned that common software,
technology providers and infrastructure create correlated exposures across
financial services.
If AI identifies
vulnerabilities faster than institutions can patch them, simultaneous
disruption could spread through payments, trading, clearing and settlement.
The Bank for
International Settlements has reached a similarly uncomfortable conclusion.
Frontier models
can autonomously identify critical vulnerabilities, develop exploits and
conduct increasingly complex cyber operations, reducing the expertise, time and
resources required for sophisticated attacks.
Critically, the
BIS argues that although AI strengthens cyber defense too, the economics may
favor attackers.
This begins to
substantiate another of Spielmann’s warnings: competitive pressure makes
restraint extraordinarily difficult. AI development increasingly resembles a
race in which organizations cannot easily slow down because competitors may
continue accelerating.
The same tension
is now visible among AI’s architects. Anthropic
CEO Dario Amodei has called for stronger safeguards and international
coordination, while OpenAI CEO Sam Altman has supported independent oversight.
Yet industry
leaders remain divided over how far regulation should go and who should impose
it.