Main Content

varbacktest

Create varbacktest object to run suite of value-at-risk (VaR) backtests

expand all in page

Description

The general workflow is:

  1. Load or generate the data for the VaR backtesting analysis.

  2. Create a varbacktest object. For more information, see Create varbacktest.

  3. Use the summary function to generate a summary report for the given data on the number of observations and the number of failures.

  4. Use the runtests function to run all tests at once.

  5. For additional test details, run the following individual tests:

    • tl — Traffic light test

    • bin — Binomial test

    • pof — Proportion of failures

    • tuff — Time until first failure

    • cc — Conditional coverage mixed

    • cci — Conditional coverage independence

    • tbf — Time between failures mixed

    • tbfi — Time between failures independence

  6. You can use the following functions to perform varbacktest enhancements:

    • append— Add portfolio and VaR data to VaR backtesting objects

    • exceptions— Format exceptions in VaR backtesting objects

    • plot— Visualize VaR portfolio data and highlight exceptions

    • select— Select VaR data from backtesting objects

    For more information, see VaR Backtesting Workflow.

Creation

Syntax

vbt = varbacktest(PortfolioData,VaRData)
vbt = varbacktest(___,Name,Value)

Description

vbt = varbacktest(PortfolioData,VaRData) creates a varbacktest (vbt) object using portfolio outcomes data and corresponding value-at-risk (VaR) data. The vbt object has the following properties:

  • PortfolioData — NumRows-by-1 numeric array containing a copy of the PortfolioData

  • VaRData — NumRows-by-NumVaRs numeric array containing a copy of the VaRData

  • PortfolioID — String containing the PortfolioID

  • VaRID — 1-by-NumVaRs string vector containing the VaRIDs for the corresponding columns in VaRData

  • VaRLevel — 1-by-NumVaRs numeric array containing the VaRLevels for the corresponding columns in VaRData.

Note

  • The required input arguments for PortfolioData and VaRData must all be in the same units. These arguments can be expressed as returns or as profits and losses. There are no validations in the varbacktest object regarding the units of these arguments.

  • If there are missing values (NaNs) in the data for PortfolioData or VaRData, the row of data is discarded before applying the tests. Therefore, a different number of observations are reported for models with different number of missing values. The reported number of observations equals the original number of rows minus the number of missing values. To determine if there are discarded rows, use the 'Missing' column of the summary report.

example

vbt = varbacktest(___,Name,Value) sets Properties using name-value pairs and any of the arguments in the previous syntax. For example, vbt = varbacktest(PortfolioData,VaRData,'PortfolioID','Equity100','VaRID','TotalVaR','VaRLevel',.99). You can specify multiple name-value pairs as optional name-value pair arguments.

example

Input Arguments

expand all

Portfolio outcomes data, specified as a NumRows-by-1 numeric array, NumRows-by-1 table, or a NumRows-by-1 timetable with a numeric column containing portfolio outcomes data. PortfolioData input sets the PortfolioData property.

Note

The required input arguments for PortfolioData and VaRData must all be in the same units. These arguments can be expressed as returns or as profits and losses. There are no validations in the varbacktest object regarding the units of these arguments.

Data Types: double | table | timetable

Value-at-risk (VaR) data, specified using a NumRows-by-NumVaRs numeric array, NumRows-by-NumVaRs table, or a NumRows-by-NumVaRs timetable with numeric columns. VaRData input sets the VaRData property.

If VaRData has more than one column (NumVaRs> 1), the PortfolioData is tested against each column in VaRData. By default, a 0.95 VaR confidence level is used for all columns in VaRData. (Use VaRLevel to specify different VaR confidence levels.)

The convention is that VaR is a positive amount. Therefore, a failure is recorded when the loss (the negative of the portfolio data) exceeds the VaR, that is, when

 -PortfolioData > VaRData

For example, a VaR of 1,000,000 (positive) is violated whenever there is an outcome worse than a 1,000,000 loss (the negative of the portfolio outcome, or loss, is larger than the VaR).

Negative VaRData values are allowed, however negative VaR values indicate a highly profitable portfolio that cannot lose money at the given VaR confidence level. That is, the worst-case scenario at the given confidence level is still a profit.

Note

The required input arguments for PortfolioData and VaRData must all be in the same units. These arguments can be expressed as returns or as profits and losses. There are no validations in the varbacktest object regarding the units of these arguments.

Data Types: double | table | timetable

Name-Value Arguments

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Before R2021a, use commas to separate each name and value, and enclose Name in quotes.

Example: vbt = varbacktest(PortfolioData,VaRData,'PortfolioID','Equity100','VaRID','TotalVaR','VaRLevel',.99)

User-defined ID for PortfolioData input, specified as the comma-separated pair consisting of 'PortfolioID' and a character vector or string. The PortfolioID name-value pair argument sets the PortfolioID property.

If PortfolioData is a numeric array, the default value for PortfolioID is 'Portfolio'. If PortfolioData is a table, PortfolioID is set by default to the corresponding variable name in the table.

Data Types: char | string

VaR identifier for VaRData columns, specified as the comma-separated pair consisting of 'VaRID' and a character vector or string. Multiple VaRIDs are specified using a 1-by-NumVaRs (or NumVaRs-by-1) cell array of character vectors or string vector with user-defined IDs for the VaRData columns. The VaRID name-value pair argument sets the VaRID property.

If NumVaRs = 1, the default value for VaRID is 'VaR'. If NumVaRs > 1, the default value is 'VaR1', 'VaR2', and so on. If VaRData is a table, 'VaRID' is set by default to the corresponding variable names in the table.

Data Types: char | cell | string

VaR confidence level, specified as the comma-separated pair consisting of 'VaRLevel' and a numeric between 0 and 1 or a 1-by-NumVaRs numeric array with values between 0 and 1 for the corresponding columns in VaRData. The VaRLevel name-value pair argument sets the VaRLevel property.

Data Types: double

Time points for each PortfolioData and VaRData entry, specified as a numeric, datetime, or duration vector. If not specified, Time defaults to 1:NumRows. If PorfolioData is a timetable, Time defaults to the RowTimes of that table.

Data Types: double | datetime | duration

Properties

expand all

Portfolio data for the VaR backtesting analysis, specified as a NumRows-by-1 numeric array containing a copy of the portfolio data.

Data Types: double

VaR data for the VaR backtesting analysis, specified as a NumRows-by-NumVaRs numeric array containing a copy of the VaR data.

Data Types: double

Portfolio identifier, specified as a string.

Data Types: string

VaR identifier, specified as a 1-by-NumVaRs string array containing the VaR IDs for the corresponding columns in VaRData.

Data Types: string

VaR level, specified as a 1-by-NumVaRs numeric array containing the VaR levels for the corresponding columns in VaRData.

Data Types: double

Time points for each PortfolioData and VaRData entry, specified as a numeric, datetime, or duration vector. If not specified, Time defaults to 1:NumRows. If PorfolioData is a timetable, Time defaults to the RowTimes of that table.

Data Types: double | datetime | duration

varbacktest PropertySet or Modify Property from Command Line Using varbacktest Modify Property Using Dot Notation
PortfolioDataYesNo
VaRDataYesNo
PortfolioIDYesYes
VaRIDYesYes
VaRLevelYesYes
TimeYesYes

Object Functions

tlTraffic light test for value-at-risk (VaR) backtesting
binBinomial test for value-at-risk (VaR) backtesting
pofProportion of failures test for value-at-risk (VaR) backtesting
tuffTime until first failure test for value-at-risk (VaR) backtesting
ccConditional coverage mixed test for value-at-risk (VaR) backtesting
cciConditional coverage independence test for value-at-risk (VaR) backtesting
tbfTime between failures mixed test for value-at-risk (VaR) backtesting
tbfiTime between failures independence test for value-at-risk (VaR) backtesting
summaryReport on varbacktest data
runtestsRun all tests in varbacktest
appendAdd portfolio, value-at-risk (VaR), and expected shortfall (ES) data to backtest objects
exceptionsFormat exceptions in value-at-risk (VaR) or expected shortfall (ES) backtest objects
plotVisualize value-at-risk (VaR) or expected shortfall (ES) and portfolio data, and highlight exceptions
selectSelect value-at-risk (VaR) or expected shortfall (ES) data from backtest objects

Examples

collapse all

Open Live Script

varbacktest takes in portfolio outcomes data and corresponding value-at-risk (VaR) data and returns a varbacktest object.

Create a varbacktest object. 

 load VaRBacktestData
 vbt = varbacktest(EquityIndex,Normal95)
vbt = 
  varbacktest with properties:

    PortfolioData: [1043x1 double]
          VaRData: [1043x1 double]
             Time: [1043x1 double]
      PortfolioID: "Portfolio"
            VaRID: "VaR"
         VaRLevel: 0.9500

vbt, the varbacktest object, contains a copy of the given portfolio data (PortfolioData property), the given VaR data (VaRData property) and all combinations of portfolio ID, VaR ID, and VaR level to be tested (PortfolioID, VaRID, and VaRLevel properties).

Run the tests using the vbt object. 

 runtests(vbt)
ans=1×11 table
    PortfolioID    VaRID    VaRLevel     TL       Bin       POF       TUFF       CC       CCI       TBF       TBFI 
    ___________    _____    ________    _____    ______    ______    ______    ______    ______    ______    ______

    "Portfolio"    "VaR"      0.95      green    accept    accept    accept    accept    accept    reject    reject

Change the PortfolfioID and VaRID properties using dot notation.

vbt.PortfolioID = 'S&P'
vbt = 
  varbacktest with properties:

    PortfolioData: [1043x1 double]
          VaRData: [1043x1 double]
             Time: [1043x1 double]
      PortfolioID: "S&P"
            VaRID: "VaR"
         VaRLevel: 0.9500


vbt.VaRID = 'Normal at 95%'
vbt = 
  varbacktest with properties:

    PortfolioData: [1043x1 double]
          VaRData: [1043x1 double]
             Time: [1043x1 double]
      PortfolioID: "S&P"
            VaRID: "Normal at 95%"
         VaRLevel: 0.9500

Run all tests using the updated varbacktest object. 

 runtests(vbt)
ans=1×11 table
    PortfolioID         VaRID         VaRLevel     TL       Bin       POF       TUFF       CC       CCI       TBF       TBFI 
    ___________    _______________    ________    _____    ______    ______    ______    ______    ______    ______    ______

       "S&P"       "Normal at 95%"      0.95      green    accept    accept    accept    accept    accept    reject    reject
Open Live Script

Create a varbacktest object. 

 load VaRBacktestData
 vbt = varbacktest(EquityIndex,Normal95)
vbt = 
  varbacktest with properties:

    PortfolioData: [1043x1 double]
          VaRData: [1043x1 double]
             Time: [1043x1 double]
      PortfolioID: "Portfolio"
            VaRID: "VaR"
         VaRLevel: 0.9500

vbt, the varbacktest object, contains a copy of the given portfolio data (PortfolioData property), the given VaR data (VaRData property) and all combinations of portfolio ID, VaR ID, and VaR level to be tested (PortfolioID, VaRID, and VaRLevel properties).

Run the tests using the varbacktest object. 

 runtests(vbt)
ans=1×11 table
    PortfolioID    VaRID    VaRLevel     TL       Bin       POF       TUFF       CC       CCI       TBF       TBFI 
    ___________    _____    ________    _____    ______    ______    ______    ______    ______    ______    ______

    "Portfolio"    "VaR"      0.95      green    accept    accept    accept    accept    accept    reject    reject

Change the PortfolfioID and VaRID properties using dot notation. 

vbt.PortfolioID = 'S&P'
vbt = 
  varbacktest with properties:

    PortfolioData: [1043x1 double]
          VaRData: [1043x1 double]
             Time: [1043x1 double]
      PortfolioID: "S&P"
            VaRID: "VaR"
         VaRLevel: 0.9500


vbt.VaRID = 'Normal at 95%'
vbt = 
  varbacktest with properties:

    PortfolioData: [1043x1 double]
          VaRData: [1043x1 double]
             Time: [1043x1 double]
      PortfolioID: "S&P"
            VaRID: "Normal at 95%"
         VaRLevel: 0.9500

Run all tests using the updated varbacktest object. 

 runtests(vbt)
ans=1×11 table
    PortfolioID         VaRID         VaRLevel     TL       Bin       POF       TUFF       CC       CCI       TBF       TBFI 
    ___________    _______________    ________    _____    ______    ______    ______    ______    ______    ______    ______

       "S&P"       "Normal at 95%"      0.95      green    accept    accept    accept    accept    accept    reject    reject
Open Live Script

Create a varbacktest object that has multiple VaR identifiers with different confidence levels. 

load VaRBacktestData
    vbt = varbacktest(EquityIndex,...
       [Normal95 Normal99 Historical95 Historical99 EWMA95 EWMA99],...
       'PortfolioID','Equity',...
       'VaRID',{'Normal95' 'Normal99' 'Historical95' 'Historical99' 'EWMA95' 'EWMA99'},...
       'VaRLevel',[0.95 0.99 0.95 0.99 0.95 0.99]);

Run the summary report for the varbacktest object. 

 summary(vbt)
ans=6×10 table
    PortfolioID        VaRID         VaRLevel    ObservedLevel    Observations    Failures    Expected    Ratio     FirstFailure    Missing
    ___________    ______________    ________    _____________    ____________    ________    ________    ______    ____________    _______

     "Equity"      "Normal95"          0.95         0.94535           1043           57        52.15       1.093         58            0   
     "Equity"      "Normal99"          0.99          0.9837           1043           17        10.43      1.6299        173            0   
     "Equity"      "Historical95"      0.95         0.94343           1043           59        52.15      1.1314         55            0   
     "Equity"      "Historical99"      0.99         0.98849           1043           12        10.43      1.1505        173            0   
     "Equity"      "EWMA95"            0.95         0.94343           1043           59        52.15      1.1314         28            0   
     "Equity"      "EWMA99"            0.99         0.97891           1043           22        10.43      2.1093        143            0

Run all tests using the varbacktest object. 

 runtests(vbt)
ans=6×11 table
    PortfolioID        VaRID         VaRLevel      TL       Bin       POF       TUFF       CC       CCI       TBF       TBFI 
    ___________    ______________    ________    ______    ______    ______    ______    ______    ______    ______    ______

     "Equity"      "Normal95"          0.95      green     accept    accept    accept    accept    accept    reject    reject
     "Equity"      "Normal99"          0.99      yellow    reject    accept    accept    accept    accept    accept    accept
     "Equity"      "Historical95"      0.95      green     accept    accept    accept    accept    accept    reject    reject
     "Equity"      "Historical99"      0.99      green     accept    accept    accept    accept    accept    accept    accept
     "Equity"      "EWMA95"            0.95      green     accept    accept    accept    accept    accept    accept    accept
     "Equity"      "EWMA99"            0.99      yellow    reject    reject    accept    reject    accept    reject    accept

Run the traffic light test (tl) using the varbacktest object. 

 tl(vbt)
ans=6×9 table
    PortfolioID        VaRID         VaRLevel      TL      Probability      TypeI      Increase    Observations    Failures
    ___________    ______________    ________    ______    ___________    _________    ________    ____________    ________

     "Equity"      "Normal95"          0.95      green       0.77913        0.26396          0         1043           57   
     "Equity"      "Normal99"          0.99      yellow      0.97991        0.03686    0.26582         1043           17   
     "Equity"      "Historical95"      0.95      green       0.85155        0.18232          0         1043           59   
     "Equity"      "Historical99"      0.99      green       0.74996        0.35269          0         1043           12   
     "Equity"      "EWMA95"            0.95      green       0.85155        0.18232          0         1043           59   
     "Equity"      "EWMA99"            0.99      yellow      0.99952      0.0011122    0.43511         1043           22
Open Live Script

Use varbacktest with table inputs and name-value pair arguments to create two varbacktest objects and run the concatenated summary report. varbacktest uses the variable names in the table inputs as PortfolioID and VaRID. 

load VaRBacktestData
vbtE = varbacktest(DataTable(:,2),DataTable(:,3:4),'VaRLevel',[0.95 0.99]);
vbtD = varbacktest(DataTable(:,5),DataTable(:,6:7),'VaRLevel',[0.95 0.99]);
[summary(vbtE); summary(vbtD)]
ans=4×10 table
     PortfolioID           VaRID           VaRLevel    ObservedLevel    Observations    Failures    Expected     Ratio     FirstFailure    Missing
    _____________    __________________    ________    _____________    ____________    ________    ________    _______    ____________    _______

    "Equity"         "VaREquity95"           0.95         0.94343           1043           59        52.15       1.1314         28            0   
    "Equity"         "VaREquity99"           0.99         0.97891           1043           22        10.43       2.1093        143            0   
    "Derivatives"    "VaRDerivatives95"      0.95         0.95014           1043           52        52.15      0.99712          9            0   
    "Derivatives"    "VaRDerivatives99"      0.99         0.97028           1043           31        10.43       2.9722         28            0

Run all the tests and concatenate the results. 

[runtests(vbtE); runtests(vbtD)]
ans=4×11 table
     PortfolioID           VaRID           VaRLevel      TL       Bin       POF       TUFF       CC       CCI       TBF       TBFI 
    _____________    __________________    ________    ______    ______    ______    ______    ______    ______    ______    ______

    "Equity"         "VaREquity95"           0.95      green     accept    accept    accept    accept    accept    accept    accept
    "Equity"         "VaREquity99"           0.99      yellow    reject    reject    accept    reject    accept    reject    accept
    "Derivatives"    "VaRDerivatives95"      0.95      green     accept    accept    accept    accept    accept    reject    reject
    "Derivatives"    "VaRDerivatives99"      0.99      red       reject    reject    accept    reject    accept    reject    reject

Run the pof test and concatenate the results. 

 [pof(vbtE); pof(vbtD)]
ans=4×9 table
     PortfolioID           VaRID           VaRLevel     POF      LRatioPOF     PValuePOF     Observations    Failures    TestLevel
    _____________    __________________    ________    ______    __________    __________    ____________    ________    _________

    "Equity"         "VaREquity95"           0.95      accept       0.91023       0.34005        1043           59         0.95   
    "Equity"         "VaREquity99"           0.99      reject        9.8298     0.0017171        1043           22         0.95   
    "Derivatives"    "VaRDerivatives95"      0.95      accept    0.00045457       0.98299        1043           52         0.95   
    "Derivatives"    "VaRDerivatives99"      0.99      reject        26.809    2.2457e-07        1043           31         0.95

References

[1] Basel Committee on Banking Supervision, Supervisory Framework for the Use of 'Backtesting' in Conjunction with the Internal Models Approach to Market Risk Capital Requirements. January, 1996, https://www.bis.org/publ/bcbs22.htm.

[2] Christoffersen, P. "Evaluating Interval Forecasts." International Economic Review. Vol. 39, 1998, pp. 841 – 862.

[3] Cogneau, Ph. “Backtesting Value-at-Risk: How Good is the Model?" Intelligent Risk, PRMIA, July, 2015.

[4] Haas, M. "New Methods in Backtesting." Financial Engineering, Research Center Caesar, Bonn, 2001.

[5] Jorion, Ph. Financial Risk Manager Handbook. 6th Edition. Wiley Finance, 2011.

[6] Kupiec, P. "Techniques for Verifying the Accuracy of Risk Management Models." Journal of Derivatives. Vol. 3, 1995, pp. 73 – 84.

[7] McNeil, A., Frey, R., and Embrechts, P. Quantitative Risk Management. Princeton University Press, 2005.

[8] Nieppola, O. “Backtesting Value-at-Risk Models.” Helsinki School of Economics, 2009.

Version History

Introduced in R2016b

expand all

See Also

| | | | | | | | | | | |

Topics