TPRM Metrics Must Measure Risk Reduction Not Activity
- Examines why activity-based TPRM metrics often fail to demonstrate real risk reduction.
- Explores the reporting metrics that resonate most with boards and senior executives.
- Discusses how organizations can prioritize remediation efforts and allocate resources effectively.
- Highlights the growing importance of concentration risk, resilience, and fourth-party visibility.
- Explores how AI and predictive analytics will shape the future of TPRM reporting and decision-making.
Many
TPRM programmes still report volumes of assessments completed or vendors
reviewed. What are the most common activity-based metrics you see organisations
over-relying on today, and why do they often fail to demonstrate real risk
reduction?
Many TPRM
programs still rely heavily on activity metrics because they are easy to
measure, easy to report, and demonstrate that work is occurring. These metrics do matter for the actual
TPRM team work measurement. The
problem is that they often measure effort rather than effectiveness. The true metrics should reflect on the
actual risk being reduced. If we see that there were one hundred assessments
conducted in a period of time, it doesn’t actually state whether any high risk
issues were identified and remediated. States the number of assessments but
doesn’t really differentiate between the complexity, was it a low risk or a
critical service review. The high volume of assessment should be also reviewed
against the size of the team. Small team with high volumes might be an
indication of the low quality reviews. It
would be better to use percentage of high risk findings mitigated in the same
review period to show how risk was actually reduced.
Similar metric
is a number of third parties reviewed
annually – this also reflects
quantity but doesn’t reflect actual risk ratings unless there is a detailed
breakdown. Organizations can review a large number of vendors but might miss a
few most critical ones. Better to look for a percentage of high and critical third
parties reviewed in line with TPRM program requirements.
Another one is
about due diligence completion rates. This does show how much work was done,
but doesn’t reflect on effectiveness of the reviews. Due diligence process
might be just gathering the documents and not really analyzing and reviewing
the controls effectiveness. What you want to see instead is how many findings
were identified and which ones resulted in mitigating actions.
Number of
contracts reviewed is another metric that doesn’t reflect the true risk
measurement. It just shows the quantity leaving outside important part – do all
contracts contain required security, privacy, audit, insurance, etc. clauses. To measure this adequately, it is good
to reflect percentage of critical and high risk contracts with all required
clauses in place or how many
contractual gaps have been unresolved compared
to the total number of identified gaps.
The speed of the
assessment is often utilized to show how efficient the process is, however it
is not a reflective on how effective the actual process is. Speed is often
achieved by reduction of the in depth analysis of the third party service or
product. This measurement would
be efficient if the quality of the reviews is
aligned to the organizational risk appetite.
How
do you balance executive demand for concise, board-level reporting with the
complexity of third-party risk data underneath? What metrics consistently
resonate with senior leadership?
In the mature
program complexity needs to be
streamlined into easily digestible ready for decision making information. It
needs to show how much risk do we have, is it within risk appetite, whether the
risk is increasing or decreasing and where the Board attention is required.
Usually total
number of third parties, with a breakdown by all risk tiers, is good to show; open high-risk findings is another important metric
to reflect together with the remediation rates, e.g. percentage of high-risk findings remediated in a quarter which
shows the actual risk reduction.Percentage of critical third parties compared
to the total population is important indicator, as you don’t want too much critical reliance on the third
parties. Condition of critical third parties, e.g. past due for reviews, or
their health indicators around risk areas. Third party related
incidents/issues/risk acceptances are also important to add to the board level reporting. These types of metrics can help to understand the risk
exposure and act on it.
As organizations face increasing cost
pressures and larger third-party ecosystems, which TPRM metrics have become
most valuable for prioritizing remediation efforts and
allocating resources?
As the
organizations grow, their third party inventory grows as well, hence to enable
cost efficient decision making, metrics should provide visibility into
prioritization, i.e. they should answer the question on where to spend the money, time and efforts to achieve
the greatest risk reduction. Findings should be broken down by severity and
risk tier, so the management can determine which ones to invest into to achieve
highest positive business impact. Another good metric is exposure above risk
appetite. This shows what can be risk accepted vs the need for immediate
remediation. The length of open findings can expose underline issues that need
to be addressed, such as resource constraints, vendor unresponsiveness, control
failures or lack of accountability. So aging metrics help management to target
areas where the risk is escalating.
A good approach
is assessing remediation action against the estimated risk reduction, e.g. implementing a certain information security related
control would have a high risk reduction, while a program document update might
be important but would have a low risk reduction on the risk exposure
landscape.
Concentration
risk metric is also worth mentioning here, if it might help to address multiple
areas at the same time, e.g. it could show exposure to the 4th party, cloud provider, geographic
area, etc. Addressing
concentration risk exposure might carry a multiplied impact on several business
areas.
Past incidents
could also drive a remediation investment of resources, such as data breaches,
regulatory findings, service disruptions. Management would more likely put funding into actually
impacted areas than findings discovered during assessments. While repeat
findings might be an indicator of unsuccessful remediation or waste of
resources.
If resources are
limited, management should prioritize by criticality as one unassessed critical
third party might have a lot more risk exposure than numerous low risk third
parties all together. Identifying potential losses can also be the driver of
the remediation efforts to take advantage of the risk adjusted return. E.g. if
the risk exposure can cause a $5mln loss while remediation effort would cost
$100K the decision is obvious.
Vendor
segmentation efficiency is another approach where management aligns risk appetite todecreased assessment activities on low risk
third parties to concentrate limited resources on more critical third parties.
As TPRM program
matures the question switches from measuring
the number of reviews to identifying remediation activities to
reduce more risk. The latter
supports resource allocation decisions and enables expansion of third-party ecosystem within the same resource constraints.
Looking
ahead three to five years, what leading indicators do you believe will become
the most important measures of third-party risk effectiveness, beyond
traditional compliance and assessment metrics?
This would be
directly correlated to TPRM program maturity in the organization. Programs that
are not mature now, will be able to move to a more mature reporting stage as
per the suggestions above. As
TPRM space evolves and emerging capabilities on the market mature, more advanced programs will be enabled to enhance reporting. It would be
effective to have percentage of critical services meeting resiliency targets,
number of critical third parties with unmitigated single point of failure,
providing visibility on whether the service/product remains operational rather
than how many questionnaires the vendor completed. Ability to show contingency plans in
place vs critical services and if they were tested or risk accepted.
Concentration
risk metric importance will grow
overtime most likely as the business would need to focus on the entire service ecosystem and not a single
third party. Metrics like percentage of critical services/products supported by
a single provider or number or critical third parties dependent on a common fourth party,
number of third parties relying on the same cloud ecosystem. To enable this
type of reporting, fourth party or nth party reporting
needs to evolve as well. More detailed reporting of fourth
parties and mapping will provide more visibility across supply chain. Metrics should enable the organization
to ensure it’s resiliency aligned with the risk appetite.
With
increased adoption of AI and continuous monitoring capabilities, how do you see
TPRM reporting evolving from retrospective risk reporting to predictive risk
intelligence for boards and executives?
AI capabilities
would change the landscape from reporting what happened to more predictive
analysis of what might happen next to enable management decision making more
efficiently. The reporting metrics would evolve into predictive risk analytics
moving decisions from reactive approach to the proactive/preventive one.
AI will also
bring the ability to see interconnected risks, e.g. connection between
financial risk, information security risk and operational risk reflecting on how the high employee
turnover and financial performance decline are
impacting investment in information security, which might be an indicator to a
potential operation disruption down the road. We probably will see more
alignment with the risk appetite and a probability of failures, such as a low
probability but high impact disruption associated with a certain provider.
Once AI tools
are trained and able to produce advanced metrics, the tools will more likely
evolve into predictive intelligence, providing
a recommendation on what would be a good approach to address the risk exposure,
which will bring additional value to the Board decision making.
Lead of Third Party Risk Management, StoneX Group Inc
Olga Baldwin is the Lead of Third Party Risk Management (TPRM) at StoneX Group Inc, a Fortune 100 company. With over 18 years of experience in TPRM, she oversees the development and execution of a comprehensive global Third Party Risk Management program. Before joining StoneX, Olga managed TPRM programs at multiple banks and gained valuable experience in the insurance and airline industries. She has extensive expertise in establishing TPRM and Vendor Risk Management (VRM) programs, system implementations, change management, regulatory examinations, and audits. Olga earned her BA from CUNY's Hunter College and an MBA from CUNY's Zicklin School of Business.
LinkedIn profile