Resolution matrix & subpixel structure

[segasai]

Hi,

Given your interest in the resolution matrix discussion in the desispec repo, I thought I'd bring to your attention what I have tried to do with the resolution matrix for stellar science, which I think it also have relevance for fastspecfit.

Resolution_matrix.pdf

(I shared this on slack a while ago as well)

Specifically this is an attempt to correctly deal with the subpixel structure in spectrum when doing resolution matrix convolutions by first preconvolving all template spectra with the fixed Gaussian kernel, and then applying the 'modified' resolution matrix.
This is what I implemented in rvspecfit https://github.com/segasai/rvspecfit/blob/e0d2b1d84d421294f5c936633415c9760ca055b5/py/rvspecfit/desi/desi_fit.py#L608
but the enabling of this was derailed by the discovery of the issue reported in the desispec repo.

From my experiments with spectral perfectionism on toy highly structured spectra https://colab.research.google.com/drive/10CZ0TkRCDuvzc3-zNx47U9d7S_VVCK0j?usp=sharing this produces correct models of extracted spectra (the current approaches do not if the true spectrum is highly structured)

Obviously fastspecfit is for the galaxies spectra, so this has less relevance for you, than for the MW, but I thought it still could be of interest, as this code is one of the direct users of the resolution matrices.