How to add points to a trained GP?

I am using the function fitrgp to fit Gaussian-Processes in the context of Bayesian optimisation (I know Matlab also has a function bayesopt, but this is to implement a different algorithm from a research paper).

Fitting works fine for a given set (X,y). During optimisation though, new points are acquired by evaluating the objective function, and I do not want to retrain the GP every time this happens; nevertheless, I would like to consider these new points for the predictions.

In short, I would like to "add" points to my existing instance of RegressionGP, using the currently fitted parameters. To illustrate this, here is a simple example of what I am trying to do:

% load example data
data = load('fisheriris');

% fit a GP using the first 100 points
obj = fitrgp( data.meas(1:100,1:3), data.meas(1:100,4) );

% add the remaining point (this is not allowed and triggers an error)
obj.X = data.meas(:,1:3);
obj.Y = data.meas(:,4);

I imagine I could create a new instance of RegressionGP using the option 'FitMethod', 'none', and make sure that all other parameters are transferred from the fitted instance, to the new one. However this is quite tedious because I have to figure out the correspondence between the class properties of RegressionGP, and the options of fitrgp; this is error-prone, and feels like a hack.

Is there a proper way to update the properties X and Y of a RegressionGP instance, without triggering a re-fitting?

Answer 1

[Nitin Khola]

Hi Jonathan,

You are on the right track. I agree that this can be very tedious. Figuring out the right parameters to pass to the subsequent "fits" after the first one can be fairly complex especially if non-default values are used. We have created a feature request based on your question so that we can consider delivering this functionality in the future.

For now, however, if we are using the default parameters for fitrgp, we need to be passing over, Beta, Sigma and KernelParameters to the new model. I would like to illustrate how we can update a model, using a simple API.

%%train and update
rng(0,'twister'); % For reproducibility
n = 1000;
x = linspace(-10,10,n)';
y = 1 + x*5e-2 + sin(x)./x + 0.2*randn(n,1);
gprMdl = fitrgp(x(1:500),y(1:500)); % use half of the data for training
gprMdl2 = updateGPRMdl(gprMdl, x,y); % now pass the entire data

%%predict from the older mdl
[ypred, ~, yci] = predict(gprMdl,x);

subplot(2,1,1);
title('no data, not much confidence on the right');
hold on;
plot(gprMdl.Y,'r.');
plot(ypred);
plot(yci(:,1),'k:');
plot(yci(:,2),'k:');
%%predict from the updated mdl
[ypred, ~, yci] = predict(gprMdl2,x);

subplot(2,1,2);
title('more confident prediction after adding data');
hold on;
plot(gprMdl2.Y,'r.');
plot(ypred);
plot(yci(:,1),'k:');
plot(yci(:,2),'k:');

%%api to update mdl
function newmdl = updateGPRMdl(mdl,Xtrain, Ytrain) % a very simple updating api, assumes defaults, will need to be modified for nondefaults
kernelparams = mdl.KernelInformation.KernelParameters; 
inisigma = mdl.Sigma;
beta = mdl.Beta; 
newmdl = fitrgp(Xtrain, Ytrain, 'FitMethod', 'none', 'Sigma', inisigma, 'Beta', beta, 'KernelParameters', kernelparams); 
end

The above code should be saved as a script before running.

HTH, Nitin

[Shrinivas Chimmalgi]

Hi Nitin,

Was this feature implemented? Unfortunately I cannot find it in the Statistics and Machine Learning toolbox.

It will allow for models to be built iteratively in an efficient manner using uncertainty sampling.

Regards, Shrinivas