Skip to content

Content Library

Back to Articles

Article May 27, 2026

Solving a Decade-Old Blind Spot in Bank Capital Models

Solving a Decade-Old Blind Spot in Bank Capital Models

Krishan Sharma, SVP, Model Risk - Regulatory Stress Testing and Capital Forecasting, CitiLinkedIn profile for Krishan Sharma

A new dual backtesting framework developed by Krishan Kumar Sharma aims to solve a longstanding weakness in bank capital modeling by isolating scenario-driven forecasting errors from downstream model inaccuracies. The methodology could improve capital eff

  • New dual backtesting framework separates scenario-driven forecasting errors from downstream model errors in bank capital models
  • Developed by Citigroup executive Krishan Kumar Sharma and published in the Journal of Risk Model Validation
  • Framework applies to CCAR, CECL, IFRS 9, and ICAAP capital regimes
  • Methodology may reduce excessive reserves and capital buffers created by uncertainty
  • Reported implementation outcomes include reserve reductions of 10 – 15 percent and capital reductions of 8 – 10 percent
  • Framework aligns with Federal Reserve SR 26-2 guidance on model validation and governance

A new diagnostic construction extends existing backtesting literature into the domain of scenario-driven regulatory capital models - with the potential, according to the author, to improve the efficiency of regulatory capital across the US banking system.

In the following article, we explore a methodological development that addresses a challenge identified by supervisors in large-bank capital modeling: the difficulty of separating, measuring, and governing errors that compound as macroeconomic scenarios cascade through downstream revenue and loss models.

The framework, developed by Krishan Kumar Sharma, a model risk management leader and senior vice president at Citigroup in New York, and published in the Journal of Risk Model Validation, introduces a dual backtesting framework - a diagnostic construction that extends existing backtesting literature - and applies it to the problem of disentangling forecasting error from downstream model error in regulatory capital exercises.

The Problem: A Persistent Blind Spot in Scenario-Driven Capital Modeling

Every major US capital regime - CCAR, CECL, IFRS 9, and ICAAP - rests on a multi-stage forecasting cascade.

External macroeconomic scenarios (from Blue Chip, Consensus Economics, or regulatory narratives) are expanded into regional and sector-specific macroeconomic variables, transformed (logs, quarter-over-quarter growth rates, lagged specifications), and finally fed into downstream Pre-Provision Net Revenue (PPNR) and Credit Loss Forecasting (CLF) models that drive reserves and regulatory capital.

The industry has long understood, in the abstract, that errors introduced at any stage do not simply add up - they may compound nonlinearly as they propagate through convex PD curves, lifetime loss projections, and capital adequacy calculations.

What has been less developed is a practical and widely implemented method to isolate which layer of the cascade contributes to which portion of the error.

Traditional validation practice, as documented in supervisory guidance and in the academic literature, typically treats each model in isolation. Validators backtest downstream PPNR and CLF models against actual macroeconomic outcomes and may not consistently challenge the scenario-generation and MEV-expansion layers upstream.

The consequence is a structural limitation: institutions may not be able to determine whether a model is miscalibrated or whether it is being influenced by a flawed scenario.

The industry’s response has often included the use of overlays, qualitative reserves, and capital buffers to absorb uncertainty that existing validation approaches have not been designed to isolate.

The cost of this conservatism, borne by shareholders and potentially affecting lending capacity, can be difficult to quantify precisely.

The Innovation: A Two-Stage Diagnostic Construction

Sharma’s contribution is a unified two-stage backtesting protocol that, taken together, enables the scenario-layer component of nested model error to be estimated separately:

Single-Blind Backtesting (SBBT). The production PPNR and CLF models are run on actual, realized historical macroeconomic paths - not on forecasts.

This isolates the intrinsic accuracy of the downstream models themselves, excluding noise contributed by the scenario layer. It addresses the question: if the bank had perfect foresight into macroeconomic conditions, how accurate would its models be?

Double-Blind Backtesting (DBBT). The full production environment is reconstructed as of a past point in time - using only the information, scenario outputs, and model versions that were in place - and the entire cascade is executed end-to-end against realized outcomes.

This captures the full chain of nested errors from scenario generation through to reserves and capital.

The framework derives an estimate of scenario-driven error by comparing the results of DBBT and SBBT. This enables a decomposition of what would otherwise appear as a combined error term into two components: scenario-related and model-related.

This decomposition is consistent with the “use test” principle articulated in the Federal Reserve’s SR 26-2  Supervisory Guidance on Model Risk Management, which requires models to be validated in the context of their intended business use.

In the peer-reviewed paper, the framework is demonstrated through a full implementation example with quantified error metrics, including Cumulative Percentage Error, Mean Absolute Percentage Error, and Percentage Error at each stage.

In a representative US mortgage Net Credit Loss example, the paper shows that single-blind and double-blind errors diverge materially, with the difference attributable to the macroeconomic scenario layer and therefore potentially addressable through targeted improvements in scenario generation.

Application in Practice: Reported Implementation Outcomes

The framework has been designed for practical application and, according to the author, has been applied in practice at US banking institutions. Materials provided by the author include industry expert commentary on adoption and utility. Based on those materials, the following outcomes have been reported:

  1. Reductions on the order of 10 to 15 percent in total reserves for the portfolios validated under the framework
  2. Reductions on the order of 8 to 10 percent in capital requirements for those portfolios
  3. Identification and correction of inconsistent assumptions along the model cascade

Confidentiality and regulatory restrictions apply. The identities of implementing institutions and entity-specific outcomes are protected under Confidential Supervisory Information (CSI) designations maintained by the Federal Financial Institutions Examination Council (FFIEC) and applicable provisions of the Dodd–Frank Act.

Events you may be interested in

View all of our upcoming events

Sign Up to Our Newsletter

Sign up

Connect Plus

Unlock more with Connect Plus membership

This content is available to Connect Plus members.