Gaussian Implementation

[dannys4]

Questions I'm chewing on regarding the Gaussian reference distribution implementation. Need expert opinion from those who work with covariance matrices and work on transport problems:

• Can I get away with using a traditional $LL^T$ Cholesky factorization for our generic gaussian distribution, or should I worry about the degenerate case and use the $LDL^T$ factorization?
• How robust should our generic Gaussian distribution be?
• Will we ever use a reference Gaussian with a dense/unstructured covariance matrix large enough for this to make a big difference when using transport maps?
• Do we make a super robust fancy generic Gaussian class and then have a special standard normal subclass, or just make a throwaway and inefficient Gaussian class since we only care about the isotropic Gaussian subclass?

Any and all thoughts/advice are appreciated!

[rubiop]

If we use the Gaussian distribution as reference I think it will be 99% the standard normal so we should be fine with whatever. For other applications of Gaussian distributions (e.g. as a prior for Bayesian inference) I think previous code used the traditional Cholesky factorization. tldr: I never used or needed fancy Gaussian implementation in my applications but maybe others did!

[mparno]

Hey Danny! For now, I would just implement a standard normal distribution. In the future, if we find that we need a multivariate Gaussian implementation, I would take a hybrid approach with options for either the fast LL^T Cholesky decomposition or an eigenvalue decomposition. In practice I've run into many situations where the covariance matrix becomes nearly singular, but you still want to draw samples. This situation is relatively common with Gaussian processes (especially those involving periodic or constant kernels) or when you're considering Gaussian posteriors stemming from observations with very little noise. In MUQ's GP stack, we first attempt an LL^T decomposition and if that fails we then compute the more expensive eigenvalue decomposition.