AggregateBayesianOptimization

Aggregate Bayesian optimization results

Since R2024b

expand all in page

Description

An AggregateBayesianOptimization object contains the aggregate results of multiple hyperparameter optimization problems that have the same optimization settings but different constraint bounds. The object is an optional output argument of any fitting function, such as fitcdiscr, that accepts the HyperparameterOptimizationOptions name-value argument.

Creation

Create an AggregateBayesianOptimization object by calling a fitting function in the list below with the OptimizeHyperparameters and HyperParameterOptimizationOptions name-value arguments and specifying to return hyperparameter optimization results. You must set ConstraintBounds and ConstraintType in the structure or HyperparameterOptimizationOptions object that you pass to the HyperParameterOptimizationOptions name-value argument of the fitting function.

For example, [Mdl,hpoResults]=fitcecoc(X,Y,OptimizeHyperparameters="all",HyperparameterOptimizationOptions=hpoObject) fits a multiclass ECOC model using the predictors X and the class labels Y, optimizes all eligible model hyperparameters using the options and constraint bounds in the HyperparameterOptimizationOptions object hpoObject, and returns the trained model Mdl and the AggregateBayesianOptimization object hpoResults.

Properties

expand all

Problem Definition Properties

This property is read-only.

Constraint function, specified as a character vector. The values of Constraint and Objective depends on the values of the ConstraintTarget and ConstraintType properties.

Value of ConstraintTargetValue of ConstraintTypeValue of ConstraintValue of Objective
"matlab""size""CompactModelSize""kfoldLoss"
"matlab""loss""kfoldLoss""CompactModelSize"
"coder""size""LearnerForCoderSize""kfoldLoss"
"coder""loss""kfoldLoss""LearnerForCoderSize"

Data Types: string

This property is read-only.

Objective function, specified as a character vector. The values of Objective and Constraint depend on the values of the ConstraintTarget and ConstraintType properties.

Value of ConstraintTargetValue of ConstraintTypeValue of ConstraintValue of Objective
"matlab""size""CompactModelSize""kfoldLoss"
"matlab""loss""kfoldLoss""CompactModelSize"
"coder""size""LearnerForCoderSize""kfoldLoss"
"coder""loss""kfoldLoss""LearnerForCoderSize"

Data Types: string

This property is read-only.

Constraint bounds, specified as an N-by-2 numeric matrix, where N is the number of optimization problems. The columns of ConstraintBounds contain the values of the lower and upper bounds on ConstraintType.

Data Types: single | double

This property is read-only.

Constraint target for the optimization problems, specified as "matlab" or "coder".

Data Types: char | string

This property is read-only.

Constraint type for the optimization problems, specified as "size" or "loss".

Data Types: char | string

Solution Properties

This property is read-only.

Minimum observed value of the objective function Objective for each optimization problem, specified as an N-by-1 numeric vector, where N is the number of rows in ConstraintBounds.

Data Types: single | double

This property is read-only.

Attained model type for each optimization problem, specified as an N-by-1 string array, where N is the number of rows in ConstraintBounds.

Data Types: string

This property is read-only.

Attained constraint value for each optimization problem, specified as an N-by-1 numeric vector, where N is the number of rows in ConstraintBounds. Each row of ConstraintAtMinObjective contains the value of Constraint at the minimum observed value of the objective function (Objective), for the corresponding constraint bounds in ConstraintBounds.

Data Types: single | double

This property is read-only.

Constraint bounds indicator for each optimization problem, specified as an N-by-1 logical array, where N is the number of rows in ConstraintBounds. An element in ConstraintBoundsAreSatisfied has a value of 1 if the final model satisfies the corresponding constraint bounds in ConstraintBounds, and 0 otherwise.

Data Types: table

This property is read-only.

Indicator of model feasibility for each optimization problem, specified as an N-by-1 logical array, where N is the number of rows in ConstraintBounds. An element in Feasible has a value of 1 if the final model is feasible (satisfies constraints), and 0 if it is infeasible. If you create an AggregateBayesianOptimization object by calling a fit function with the hyperparameter optimization option Optimizer="bayesopt", then an optimization problem can have a ConstraintBoundsAreSatisfied value of 1 and a Feasible value of 0. This result can occur if the final attained point satisfies the constraint bounds in ConstraintBounds but lies outside the optimization confidence bounds (see bestPoint).

Data Types: logical

This property is read-only.

Hyperparameters used in each optimization problem, specified as an N-by-1 cell array of optimizableVariable objects. If you create an AggregateBayesianOptimization object using fitcauto or fitrauto, then the size of each optimizableVariable object is 1-by-K, where K depends on the model type and hyperparameter optimization options. Otherwise, the size of each optimizableVariable object is K-by-1.

Data Types: cell array

This property is read-only.

Hyperparameter optimization results for each optimization problem, specified as an N-by-1 cell array, where N is the number of rows in ConstraintBounds. If the structure or HyperParameterOptimizationOptions object that you pass to a fitting function has the specification Optimizer="bayesopt" (the default), then the elements of HyperParameterOptimizationResults are BayesianOptimization objects. Otherwise, the elements are tables of hyperparameters.

If you create an AggregateBayesianOptimization object by calling a fit function with the hyperparameter optimization options SaveIntermediateResults=true and a ConstraintBounds value that is not [], then an additional iteration is stored in the BayesianOptimization objects (or tables). During this iteration, which is not shown in the command-line output of the fit function, the software evaluates the constraint and objective functions, but does not perform any optimization or model training.

Data Types: cell array

This property is read-only.

Optimization results for each problem, specified as an N-by-1 cell array of BayesianOptimization objects, where N is the number of rows in ConstraintBounds. If the HyperParameterOptimizationOptions object that you pass to a fitting function has the specification Optimizer="bayesopt" (the default), then the contents of BayesianOptimizationResults and HyperparameterOptimizationResults are the same.

If you create an AggregateBayesianOptimization object by calling a fit function with the hyperparameter optimization options SaveIntermediateResults=true and a ConstraintBounds value that is not [], then an additional iteration is stored in the BayesianOptimization objects. During this iteration (which is not displayed in the command-line output of the fit function), the software evaluates the constraint and objective functions, but does not perform any optimization or model training.

Data Types: cell array

Object Functions

plotPlot aggregated hyperparameter optimization results
resumeResume hyperparameter optimization problems
summarySummary table for AggregateBayesianOptimization object

Examples

collapse all

Open Live Script

Repeat the hyperparameter optimization of a trained classification tree model several times using the same optimization settings, but with different constraint bounds each time.

Load the ionosphere data set. 

load ionosphere.mat

Create a HyperparameterOptimizationOptions object that contains the settings for three hyperparameter optimization problems. For each problem, specify to use the compact size of the trained model as the constraint, and use the default settings for the other optimization options. Specify the constraint bounds as 0 to 5000 bytes for the first problem, 0 to 20000 bytes for the second problem, and 0 to 50000 bytes for the third problem.

rng(0,"twister"); % For reproducibility
hpoOptions = hyperparameterOptimizationOptions(ConstraintType="size", ...
    ConstraintBounds=[5000; 20000; 50000])
hpoOptions = 
  HyperparameterOptimizationOptions with properties:

                  Objective: "kfoldLoss"
                 Constraint: "CompactModelSize"
    AcquisitionFunctionName: "expected-improvement-per-second-plus"
           ConstraintBounds: [3×2 double]
           ConstraintTarget: "matlab"
             ConstraintType: "size"
                      KFold: 5
                    MaxTime: Inf
                  Optimizer: "bayesopt"
                Repartition: 0
    SaveIntermediateResults: 0
                  ShowPlots: 1
                UseParallel: 0
                    Verbose: 1

hpoOptions is a HyperparameterOptimizationOptions object that contains hyperparameter optimization options for the classification tree fitting function.

Modify the default property values in hpoOptions to suppress the display of plots and command-line output.

hpoOptions.ShowPlots=false;
hpoOptions.Verbose=0;

Train a classification tree model using the fitctree function, and optimize the model hyperparameters for each optimization problem subject to the constraints in hpoOptions. Additionally return the results object hpoResults.

[Mdl,hpoResults]=fitctree(X,Y,OptimizeHyperparameters="auto", ...
    HyperparameterOptimizationOptions=hpoOptions)
Mdl=3×1 cell array
    {1×1 ClassificationTree}
    {1×1 ClassificationTree}
    {1×1 ClassificationTree}


hpoResults = 
  AggregateBayesianOptimization with properties:

                            Objective: "kfoldLoss"
                           Constraint: "CompactModelSize"
          BayesianOptimizationResults: {3×1 cell}
             ConstraintAtMinObjective: [3×1 double]
                     ConstraintBounds: [3×2 double]
         ConstraintBoundsAreSatisfied: [3×1 logical]
                     ConstraintTarget: "matlab"
                       ConstraintType: "size"
                             Feasible: [3×1 logical]
    HyperparameterOptimizationResults: {3×1 cell}
                LearnerAtMinObjective: [3×1 string]
                         MinObjective: [3×1 double]
                 VariableDescriptions: {3×1 cell}

Mdl is a cell array that contains the trained classification model object for each optimization problem. hpoResults is an AggregateBayesianOptimization object that contains the options and results for each optimization problem.

Version History

Introduced in R2024b

See Also

| | |

Topics