Skip to main content

5.3 Ongoing Analysis

No: BCS 290 Date(g): 12/6/2006 | Date(h): 16/5/1427

Effective from Jan 01 2008 - Dec 31 2007
To view other versions open the versions tab on the right

5.3.1The second analytical support for the validity of a bank’s rating system is the ongoing analysis intended to confirm that the rating system is implemented and continues to perform as intended. Such analysis involves process verification and benchmarking.
 
 Process verification
 
5.3.2Specific verification activities depend on the rating approach. If a model is used for rating, verification requires reviewers who are independent of the model development to evaluate the soundness of the model, including the theory, assumptions and mathematical/empirical basis. In addition, the evaluation should include the assessment of the compliance with the requirements set out in subsection 4.6 of the “Minimum Requirements for Internal Rating Systems under IRB Approach” on use of models.
 
5.3.3If expert judgment is used for rating, verification requires other individual reviewers to evaluate whether the rater has followed rating policy. The minimum requirements for verification of ratings assigned by individuals are:
 
 a transparent rating process;
 
 a database with information used by the rater; and
 
 documentation of how the decisions were made.
 
5.3.4Rating process verification also includes override monitoring. The requirements for overrides are set out in subsection 5.3 of the “Minimum Requirements for Internal Rating Systems under IRB Approach”. A reporting system capturing data on reasons for overrides could facilitate learning about whether overrides improve accuracy.
 
 Benchmarking
 
5.3.5Benchmarking is a set of activities that uses alternative tools to draw inferences about the correctness of ratings before outcomes are actually known. Benchmarking of a rating system demonstrates whether another rater or rating method attaches the same rating to a particular obligor or facility. At a minimum, banks should establish a process in which a representative sample of its internal ratings is compared to third-party ratings (e.g. independent internal raters, external rating agencies, models, or other market data sources) of the same credits. Regardless of the rating approach, the benchmark can either be a judgment-based or a model based rating. Examples of such benchmarking include: rating reviewers completely re-rate a sample of credits rated by individuals in a judgment-based system; an internally developed model is used to rate credits rated earlier in a judgment-based system; individuals rate a sample of credits rated by a model; internal ratings are compared against results from external agencies or external models.
 
 Banks can also consider benchmarking which includes activities designed to draw broader inferences about whether the rating system – as opposed to individual ratings – is working as expected. Bank can look for consistency in ranking or consistency in the values of rating characteristics for similarly rated credits. Examples of such benchmarking activities include:
 
 analyzing the characteristics of obligors that have received common ratings; monitoring changes in the distribution of ratings over time;
 
 calculating a transition matrix from changes in ratings in a bank portfolio and comparing it to historical transition matrices from publicly available ratings or external data pools.
 
5.3.6If benchmarking evidence suggests a pattern of rating differences, it should lead the bank to investigate the source of the differences. Thus, the benchmarking process illustrates the possibility of feedback from ongoing validation to model development.