Backtesting Requirements
Effective from Dec 28 2022 - Dec 27 2022
To view other versions open the versions tab on the right
| 12.4 | Backtesting requirements compare the value-at-risk (VaR) measure calibrated to a one-day holding period against each of the actual P&L (APL) and hypothetical P&L (HPL) over the prior 12 months. Specific requirements to be applied at the bank-wide level and trading desk level are set out below. | |||||||||||||||||||||||||||||||||||||||||
| 12.5 | Backtesting of the bank-wide risk model must be based on a VaR measure calibrated at a 99th percentile confidence level. | |||||||||||||||||||||||||||||||||||||||||
| (1) | An exception or an outlier occurs when either the actual loss or the hypothetical loss of the bank-wide trading book registered in a day of the backtesting period exceeds the corresponding daily VaR measure given by the model. As per [16.8], exceptions for actual losses are counted separately from exceptions for hypothetical losses; the overall number of exceptions is the greater of these two amounts. | |||||||||||||||||||||||||||||||||||||||||
| (2) | In the event either the P&L or the daily VaR measure is not available or impossible to compute, it will count as an outlier. | |||||||||||||||||||||||||||||||||||||||||
| 12.6 | In the event an outlier can be shown by the bank to relate to a non-modellable risk factor, and the capital requirement for that non-modellable risk factor exceeds the actual or hypothetical loss for that day, it may be disregarded for the purpose of the overall backtesting process if SAMA is notified accordingly and does not object to this treatment. In these cases, a bank must document the history of the movement of the value of the relevant non-modellable risk factor and have supporting evidence that the non-modellable risk factor has caused the relevant loss. | |||||||||||||||||||||||||||||||||||||||||
| If the backtesting exception at a desk-level test is being driven by a non-modellable risk factor that receives an SES capital requirement that is in excess of the maximum of the APL loss or HPL loss for that day, it is permitted to be disregarded for the purposes of the desklevel backtesting. The bank must be able to calculate a non-modellable risk factor capital requirement for the specific desk and not only for the respective risk factor across all desks. For example, if the P&L for a desk is SAR –1.5 million and VaR is SAR 1 million, a nonmodellable risk factor capital requirement (at desk level) of EUR 0.8 million would not be sufficient to disregard an exception for the purpose of desk-level backtesting. The nonmodellable risk factor capital requirement attributed to the standalone desk level (without VaR) must be greater than the loss of SAR 1.5 million in order to disregard an exception for the purpose of desk-level backtesting. | ||||||||||||||||||||||||||||||||||||||||||
| 12.7 | The scope of the portfolio subject to bank-wide backtesting should be updated quarterly based on the results of the latest trading desk-level backtesting, risk factor eligibility test and PLA tests. | |||||||||||||||||||||||||||||||||||||||||
| 12.8 | The framework for SAMA interpretation of backtesting results for the bank-wide capital model encompasses a range of possible responses, depending on the strength of the signal generated from the backtesting. These responses are classified into three backtesting zones, distinguished by colours into a hierarchy of responses. | |||||||||||||||||||||||||||||||||||||||||
| (1) | Green zone. This corresponds to results that do not themselves suggest a problem with the quality or accuracy of a bank’s model. | |||||||||||||||||||||||||||||||||||||||||
| (2) | Amber zone. This encompasses results that do raise questions in this regard, for which such a conclusion is not definitive. | |||||||||||||||||||||||||||||||||||||||||
| (3) | Red zone. This indicates a result that almost certainly indicates a problem with a bank’s risk model. | |||||||||||||||||||||||||||||||||||||||||
| 12.9 | These zones are defined according to the number of exceptions generated in the backtesting programme considering statistical errors as explained in [16.9] to [16.21]. Table 1 sets out boundaries for these zones and the presumptive SAMA response for each backtesting outcome, based on a sample of 250 observations. | |||||||||||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||||||||||||
12.10 | The backtesting green zone generally would not initiate a SAMA increase in capital requirements for backtesting (ie no backtesting add-on would apply). | |||||||||||||||||||||||||||||||||||||||||
| 12.11 | Outcomes in the backtesting amber zone could result from either accurate or inaccurate models. However, they are generally deemed more likely for inaccurate models than for accurate models. Within the backtesting amber zone, SAMA will impose a higher capital requirement in the form of a backtesting add-on. The number of exceptions should generally inform the size of any backtesting add-on, as set out in Table 1 of [12.9]. | |||||||||||||||||||||||||||||||||||||||||
| 12.12 | A bank must also document all of the exceptions generated from its ongoing backtesting programme, including an explanation for each exception. | |||||||||||||||||||||||||||||||||||||||||
| 12.13 | A bank may also implement backtesting for confidence intervals other than the 99th percentile, or may perform other statistical tests not set out in this standard. | |||||||||||||||||||||||||||||||||||||||||
| 12.14 | Besides a higher capital requirement for any outcomes that place the bank in the backtesting amber zone, in the case of severe problems with the basic integrity of the model, SAMA may consider whether to disallow the bank’s use of the model for market risk capital requirement purposes altogether. | |||||||||||||||||||||||||||||||||||||||||
| 12.15 | If a bank’s model falls into the backtesting red zone, SAMA will automatically increase the multiplication factor applicable to the bank’s model or may disallow use of the model. | |||||||||||||||||||||||||||||||||||||||||
| Backtesting at the trading desk level | ||||||||||||||||||||||||||||||||||||||||||
| 12.16 | The performance of a trading desk’s risk management model will be tested through daily backtesting. | |||||||||||||||||||||||||||||||||||||||||
| 12.17 | The backtesting assessment is considered to be complementary to the PLA assessment when determining the eligibility of a trading desk for the IMA. | |||||||||||||||||||||||||||||||||||||||||
| 12.18 | At the trading desk level, backtesting must compare each desk’s one-day VaR measure (calibrated to the most recent 12 months’ data, equally weighted) at both the 97.5th percentile and the 99th percentile, using at least one year of current observations of the desk’s one-day P&L. | |||||||||||||||||||||||||||||||||||||||||
| (1) | An exception or an outlier occurs when either the actual or hypothetical loss of the trading desk registered in a day of the backtesting period exceeds the corresponding daily VaR measure determined by the bank’s model. Exceptions for actual losses are counted separately from exceptions for hypothetical losses; the overall number of exceptions is the greater of these two amounts. | |||||||||||||||||||||||||||||||||||||||||
| (2) | In the event either the P&L or the risk measure is not available or impossible to compute, it will count as an outlier. | |||||||||||||||||||||||||||||||||||||||||
| Volatility scaling of returns for VaR calculation at the discretion of the bank that result in a shorter observation period being used is not allowed. A bank may scale up the volatility of all observations for a selected (group of) risk factor(s) to reflect a recent stress period. The bank may use this scaled data to calculate future VaR and expected shortfall estimates only after ex ante notification of such a scaling to SAMA. | ||||||||||||||||||||||||||||||||||||||||||
| 12.19 | If any given trading desk experiences either more than 12 exceptions at the 99th percentile or 30 exceptions at the 97.5th percentile in the most recent 12-month period, the capital requirement for all of the positions in the trading desk must be determined using the standardised approach.47 | |||||||||||||||||||||||||||||||||||||||||
47 Desks with exposure to issuer default risk must pass a two-stage approval process. First, the market risk model must pass backtesting and PLA. Conditional on approval of the market risk model, the desk may then apply for approval to model default risk. Desks that fail either test must be capitalised under the standardised approach.