Inducing Point Kernel Performs Better than Base Kernel Alone

[kapsiak]

Hello Everyone,

I am working on a research problem, where part of the work requires interpolating a shape in to some unknown region.
The goal of course it to produce an accurate estimation of the shape in this unknown region, shown in the images here as a green outline.
I have been using Exact GPR with reasonable success, but am encountering an "issue" that I was hoping you could offer advice on.

The gist is that for pretty all kernels I have worked on, from basic RBF to DKL transformations of the inputs, the regression seems to produce better estimations when using inducing points vs using the bare kernel. The metric here being used to estimate the quality of the interpolation is the pull, the difference between the GPR mean and the known value, divided by the known uncertainty on the test data.
This particular base kernel doesn't do super well in parts of the plane, but even DKL kernels which do much better show similar behavior.

In all cases, I am using a fixed noise Gaussian likelihood, since the observed data has known uncertainties, without learning additional noise.

For the example shown here, the base kernel is just Scale(RBF), and the inducing model wraps this using NumData/2 inducing points. This behavior seems to persist across kernels we have worked with. The values here are Z-scored and the locations are normalized to a unit square.

I am quite new to GPR, so my concern is that this may be pointing so some poor modeling that I am not realizing, so I was wondering if anyone had ideas as to why this might be.

Thank you for any advice.