Newer libraries

[dannys4]

With the advent of C++20, one can use concepts and constraints, and cuda just officially supported C++20 in the past year or so in 12.xx. I'd be happy to talk about why this can improve our code from a performance perspective. Is it worth saying "We should build for C++20" and fixing any issues that may arise? In theory, this prevents us from deploying on certain machines. In practice, these machines are largely antiquated supercomputers that are both a) too old and b) too big for our targeted usecases. On the other hand, this might limit what code we can link against (there's some snarky remark about std::string lurking here). It looks like our library compiles with C++20 as-is? It's hard to tell due to the next point.

Similarly, we could be using Kokkos 4.0. Most notably, this streamlines the GPU compilation process (which is annoying, and possibly incomprehensible for a non-developer). However, there are some changes I know we need to make to maintain compliance with things that they've broken. The complicating factor is that you might need Kokkos 4.0 to make effective use of C++20 (when you use CUDA), so you can't just use the above and not bump the version of kokkos.

Just some thoughts, wanted to open discussion to anyone interested.

[mparno]

I'm likely supportive, but would like to better understand the benefits of using c++20 first. How will it improve performance? We started MUQ in 2012 just after c++11 came out and made the decision to rely on c++11 features (like smart pointers). There were a couple years where that was slightly limiting, but eventually it was no big deal and was definitely worth it in the long run. This could be a similar time for MParT.

[dannys4]

In theory, one could make a ParameterizedFunctionConcept that describes any function that has Evaluate, InputGradient, and CoeffGradient, etc. The compiler can check that any class implementing ParameterizedFunctionConcept defines functions without the need for a vtable, which can obviously be costly. In practice we wouldn't just abandon the PFB superclass! Because ParameterizedFunctionBase is facing the bindings, and interfacing bindings with concepts would be a nightmare compared to just using classes. Further, using the vtable isn't too bad for the top-level objects (though it's certainly worth investigating the overhead incurred when using MultivariateExpansion as a child class of PFB, particularly regarding the many evaluations it takes for MonotoneIntegrand). Something that we could do here though is create a concept for low-level or low-ish level things (like MultivariateExpansion) to minimize using the vtable for function calls inside of Kokkos kernels.

Another, more complicated, example: I'm still trying to work on implementing alternate 1d basis to just polynomials within MultivariateExpansionWorker, and there are some simple concepts at this level could be like QuadratureConcept, PosFuncConcept. However, for something like Basis1dConcept, the templating is somewhat vague-- I run into things like "what functions does my 1d basis function need to have in common with OrthogonalPolynomial?". And while this is reasonably easy to figure out, it's impossible to enforce besides just letting the compiler throw obtuse errors. For the earlier two concept suggestions, concepts would likely mitigate this very cleanly. However, suppose I want to use a heterogeneous basis with Hermite polynomials in some dimensions, legendre polynomials in others, and then RBFs/wavelets/sigmoids on the diagonal. How do you implement this? Well, it'd be nice to have an array/vector of basis objects on the stack. But it's tricky because they all may be different types. The solution is to make sure they all share the same base class type when creating the vector, but then you need to enforce that each basis function is dynamically allocated to prevent issues from accessing the child class implementation of overridden function ("type slicing" or "object slicing"). You could hope to mitigate this using concepts/constraints (implement a concept that's like Basis1dConcept which requires Evaluate, Derivative, SecondDerivative functions or something), but you can't make a vector of concepts-- you need all of the elements of the vector to share the same type, so then you need to employ "type erasure" to create a class that just holds some object satisfying a concept constraint. I'm mostly writing this out for future reference, because it's taken awhile to figure all this out. EDIT: Alternatively we could just not implement this at all and let the user figure out how they want to extend Basis1dConcept for heterogeneous function classes (you may even be able to get away with using parameter packs and tuples), though that's no fun.

Anyway, this is why it's complicated, but it can pay off; creating functions we call a lot that can be resolved at compile time will improve performance--I'm inclined to add "dramatically", and inclined to say that this will be even more noticeable for CUDA. On a much more benign note, I really like the new string formatting tools in C++20 (they make C++ string formatting like python).

[dannys4]

Obviously, knowing how much this would improve performance could probably be estimated by figuring the overhead of using the vtable within kernels. I'm not immediately sure how to do this, though I could look it up. I'm also open to filing this as an issue as "enhancement" and then forgetting about it for 4-6 months 😆.

[dannys4]

relevant blog post for those unfamiliar with the vtable and how it can effect performance: https://johnnysswlab.com/the-true-price-of-virtual-functions-in-c/

[mparno]

I don't think we have many (if any) virtual functions that get called a lot. I tried to be pretty careful with that early on and actually wouldn't expect much of a performance change if we removed the few remaining virtual functions we have in the top level classes, but I may be wrong. That said, I think it could be a very good idea to use a concept to formalize the requirements on 1d basis functions and classes like MultivariateExpansionWorker.

My only real concern is that we would lose support for somewhat recent but not cutting edge versions of CUDA (like what I use). I'll do some testing to see if I can mix concepts with cuda in my environment. Worst case we can wait a bit until these recent CUDA versions are more ubiquitous.

[dannys4]

Yes, something I meant to (but forgot to include) is that you do require CUDA 12, which is only available on >=Maxwell (i.e. within the past ~8ish years). I'm in no rush. I agree that the top level classes shouldn't matter, I'm mostly worried about the embedded MultivariateExpansionWorker within MonotoneIntegrand, which would get called a whole lot when, e.g., inverting a rectified polynomial component.

…

________________________________ From: Matt Parno ***@***.***> Sent: Thursday, October 26, 2023 9:41 AM To: MeasureTransport/MParT ***@***.***> Cc: Daniel Garner Sharp ***@***.***>; Author ***@***.***> Subject: Re: [MeasureTransport/MParT] Newer libraries (Discussion #356) I don't think we have many (if any) virtual functions that get called a lot. I tried to be pretty careful with that early on and actually wouldn't expect much of a performance change if we removed the few remaining virtual functions we have in the top level classes, but I may be wrong. That said, I think it could be a very good idea to use a concept to formalize the requirements on 1d basis functions and classes like MultivariateExpansionWorker. My only real concern is that we would lose support for somewhat recent but not cutting edge versions of CUDA (like what I use). I'll do some testing to see if I can mix concepts with cuda in my environment. Worst case we can wait a bit until these recent CUDA versions are more ubiquitous. — Reply to this email directly, view it on GitHub<#356 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AKJG6T5F4FU4VLHUHF56K4LYBJSCHAVCNFSM6AAAAAA6OPWZA2VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TGOJTGEZTS>. You are receiving this because you authored the thread.Message ID: ***@***.***>