| 16.1 | An additional consideration in specifying the appropriate risk measures and trading outcomes for profit and loss (P&L) attribution test and backtesting arises because the internally modelled risk measurement is generally based on the sensitivity of a static portfolio to instantaneous price shocks. That is, end-of-day trading positions are input into the risk measurement model, which assesses the possible change in the value of this static portfolio due to price and rate movements over the assumed holding period.
|
| 16.2 | While this is straightforward in theory, in practice it complicates the issue of backtesting. For instance, it is often argued that neither expected shortfall nor value-at-risk measures can be compared against actual trading outcomes, since the actual outcomes will reflect changes in portfolio composition during the holding period. According to this view, the inclusion of fee income together with trading gains and losses resulting from changes in the composition of the portfolio should not be included in the definition of the trading outcome because they do not relate to the risk inherent in the static portfolio that was assumed in constructing the value-at- risk measure.
|
| 16.3 | This argument is persuasive with regard to the use of risk measures based on price shocks calibrated to longer holding periods. That is, comparing the liquidity- adjusted time horizon 99th percentile risk measures from the internal models capital requirement with actual liquidity- adjusted time horizon trading outcomes would probably not be a meaningful exercise. In particular, in any given multiday period, significant changes in portfolio composition relative to the initial positions are common at major trading institutions. For this reason, the backtesting framework described here involves the use of risk measures calibrated to a one- day holding period. Other than the restrictions mentioned in this paper, the test would be based on how banks model risk internally.
|
| 16.4 | Given the use of one-day risk measures, it is appropriate to employ one-day trading outcomes as the benchmark to use in the backtesting programme. The same concerns about “contamination” of the trading outcomes discussed above continue to be relevant, however, even for one-day trading outcomes. That is, there is a concern that the overall one-day trading outcome is not a suitable point of comparison, because it reflects the effects of intraday trading, possibly including fee income that is booked in connection with the sale of new products.
|
| 16.5 | On the one hand, intraday trading will tend to increase the volatility of trading outcomes and may result in cases where the overall trading outcome exceeds the risk measure. This event clearly does not imply a problem with the methods used to calculate the risk measure; rather, it is simply outside the scope of what the measure is intended to capture. On the other hand, including fee income may similarly distort the backtest, but in the other direction, since fee income often has annuity-like characteristics. Since this fee income is not typically included in the calculation of the risk measure, problems with the risk measurement model could be masked by including fee income in the definition of the trading outcome used for backtesting purposes.
|
| 16.6 | To the extent that backtesting programmes are viewed purely as a statistical test of the integrity of the calculation of the risk measures, it is appropriate to employ a definition of daily trading outcome that allows for an uncontaminated test. To meet this standard, banks must have the capability to perform the tests based on the hypothetical changes in portfolio value that would occur were end-of-day positions to remain unchanged.
|
| 16.7 | Backtesting using actual daily P&Ls is also a useful exercise since it can uncover cases where the risk measures are not accurately capturing trading volatility in spite of being calculated with integrity.
|
| 16.8 | For these reasons, the Committee requires banks to develop the capability to perform these tests using both hypothetical and actual trading outcomes. In combination, the two approaches are likely to provide a strong understanding of the relation between calculated risk measures and trading outcomes. The total number of backtesting exceptions for the purpose of the thresholds in [12.9] must be calculated as the maximum of the exceptions generated under hypothetical or actual trading outcomes.
|
