How to install hydrogym on a supercomputer ?

[ReHoss]

Hi everyone, since firedrake is not conda friendly, what are the best practices to install hydrogym on a supercomputer ?
I mean there is this effort to include conda compatibility : firedrakeproject/firedrake#2543 (comment)

Otherwise, (I am new to HPC) could it be enough to work with a pip environment (the one created during firedrake installation) in the HPC node ?

Or should we use Docker ?

[jcallaham]

The best route will probably depend a little on how packages and permissions and such are managed on your cluster. But I think that probably the more difficult thing to install will be Firedrake compared to Hydrogym. Have you been able to get Firedrake working, or does there happen to be a module with it pre-installed on your cluster?

From there you can just install Hydrogym into whatever environment works for Firedrake. For instance, if you are able to get Firedrake working with conda you can just clone Hydrogym locally and then do python -m pip install -e ., which should install into your conda environment using the conda Python version.

Docker is also a good alternative, if that's something that works well on your cluster. Just depends on what the admins prefer.

[jcallaham]

That would be great if you update once you figure out some solutions - I think the permissions and such can vary quite a bit, so it might help other potential users.

For me, I haven't run it on a cluster yet. We are working on getting access for collaborators, but I've just been doing development on a workstation that has plenty of compute for normal simulations (i.e. not RL training).

[ReHoss]

The potential solutions:

• Ask to the HPC administrators to install Firedrake (the one I am following since second solution fails for now)

• Firedrake devs developed a Spack environment which does not requires super-user rights https://github.com/firedrakeproject/firedrake-spack .
  However I tried this and it failed because of a relatively minor bug on PETSc MPI linkage Installation issue: petsc firedrakeproject/firedrake-spack#28 . Moreover, I don't really know how to bind hydrogym to this Spack environments and other python libraries I would like to use...

• Use a Docker/Singularity container
  My HPC use Singularity which handle Docker containers. However I don't know how easy it is to run Job Arrays when having the program within a container, I mean do I need to duplicate the Docker image n times for n differents experiments ?
  plus I don't know if the container is able to write to my $WORKDIR database directly (I guess it is possible with port etc.?).

Thanks:)

[ludgerpaehler]

Hey,

In the update we are currently pushing up we went with Option 3 where Ray launches container instances, and Ray itself can be run in the cloud on a Kubernetes-based system, or talk to a Slurm-based cluster. You should only have to wait for ~1 week until the final state will be verified, and pushed up into the repo.

Best,
Ludger

[ReHoss]

Hi @ludgerpaehler @jcallaham , I had to change of HPC and know it becomes difficult to me to install FireDrake again on a new HPC, is there a Singularity solution or Ray available now?

[ludgerpaehler]

Can you remind me at the end of the week :) Currently working to update the docs, which should then hopefully be able to answer all your questions and if not merit more work from my side :)