Gaussian process regression - how to fit a basis function but not other parameters

I'm doing Gaussian process regression with the fitrgp function and want to manually control some parameters such as the kernel parameters and the noise parameter, but automatically fit a basis function (e.g. 'linear').

However, it seems that to fit a basis function the FitMethod parameter cannot be 'none':

1. Manual parameter settings (no parameter fitting) but no basis function

gpr_model = fitrgp(x_d, y_d, ...
    'KernelParameters', [1.5; 1], ...
    'Sigma', 0.1, ...
    'FitMethod', 'none' ...
    );

2. To fit the basis function I can't have FitMethod set to 'none' ('exact' is the default) but then it ignores my parameter values:

gpr_model = fitrgp(x_d, y_d, ...
    'KernelParameters', [1.5; 1], ...
    'Sigma', 0.1, ...
    'BasisFunction', 'linear' ...
    );

3. If I set FitMethod to none, it ignores the basis function:

gpr_model = fitrgp(x_d, y_d, ...
    'KernelParameters', [1.5; 1], ...
    'Sigma', 0.1, ...
    'BasisFunction', 'linear', ...
    'FitMethod', 'none' ...
);

(same a Fig. 1).

Is there a way to fit a basis function but not the other parameters?

[Bill Tubbs]

As a follow-up, I see that fitrgp has a ConstantSigma argument. Also there is a SigmaLowerBound. Is there any way of fixing or constraining the kernel parameters? (I can't see one). That would solve my problem I think.

[mono]

Facing a similar problem to perform optimizations of a gaussian process with one specific paramater fixed...

[Bill Tubbs]

Let me know if you find a solution!

Answer 1

[Vitek Stepien]

If I understand correctly, you're trying to programatically figure out whoch fitting function would work best for the data given? If so, then the short answer is no, there isn't a good way - I tried doing that in a much more complex setting and there is no automatic way of trying different functions. The workaround would be to write a script that fits a number of functions to the data (I presume you have a few specific functions that you wnat to check), and check which fit has the best "goodness", for example using SSE as the quantifier. Then the script would return the fucntion with best fit (lowest SSE) and you can go from there. Let me know if that helped!

[Bill Tubbs]

No not exactly. I'm trying to fit the GPR with a given basis function (linear) but holding some parameters fixed (i.e. not fitted to the data during the regression process). The reason is I am working in a low data regime so the 100% data-driven approach produces poor results as shown in the plot. That is why the linear explicit basis function is important.

In summary, I want the fixed (i.e. manual) parameter settings used in example 1 with the fitted basis function used in example 2.

[mono]

If I understood the question correctly, @Bill Tubbs and myself are trying to perform parameters optimizations but with one or several specific paramaeters fixed.

E.g. with initial theta as theta0 = [1, 2], now only want to optimizae theta0[1] but with theta0[2] = 2 fixed without optimization.

[Vitek Stepien]

I see, it looked similar to a problem I had but I guess I don't know how to help in your case. Sorry and good luck finding the answer!

Answer 2

[Alan Weiss]

I might misunderstand what you are trying to do, but perhaps you could use the OptimizeHyperparameters name-value argument to find the parameter you are trying to optimize. I am not sure that this can give you what you want, because I don't really understand what you are doing, but you can check whether the argument might be applicable.

Good luck,

Alan Weiss

MATLAB mathematical toolbox documentation