nvfortran support

[jphaupt]

I've found the nvfortran compiler mentioned a few times on here (e.g.), but I can't seem to get it to work with fpm. My understanding is that you simply do fpm build --compiler 'nvfortran'. However, with this I get the (rather unhelpful) error Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Also, related: how do I link to packages like cudafor and curand? For example, in a CUDA program to calculate pi (via Monte Carlo, say) I might have use curand_m, which I would build with (where CUDALIB is the path for curand)

nvfortran -O3 -c $(CUDALIB)/curand_m.cuf
nvfortran -O3 -o bin/pi_cuda precision_m.o curand_m.o -lcurand app/piMC_cuda.cuf

and it would be nice to be able to do this with fpm.

Finally, tangentially related but is there support for using multiple compilers, for different executables? I want to benchmark CUDA implementations against MPI implementations and it would be convenient to have them in the same fpm.toml file (otherwise, not a huge deal as I can just separate them into different folders, though they will be sharing some modules so I'm not sure this will work as I'd rather not copy the same code).

[urbanjost]

I cannot duplicate your error. How did you build or pull down your copy of fpm? If you have gfortran on your machine you might want to try building it from one of the single-file sources for fpm. If you do "fpm help build" you should see that there is a --flags option and you might want to optionally put the "curand" library in the fpm.toml "load" section. You would run the pi_cuda command with "fpm run" or install it with '"fpm install ...". There is on-going work on allowing for custom options to be specified in the fpm.toml file itself. If your files are in a public repository I can try it to see if I can find a combination that works if you have problems. The "cuf" suffix might cause an issue right now.

fpm build --compiler nvfortran --flag '-cuda -Mpreprocess'

If you are using kernel directives you will want the files processed like they were .CUF files and will need the -Mpreprocess option; fpm expects the Fortran files to end in .f90|.F90|.f|.F so you need to name them (at least currently) .f90 or .F90 and use the -cuda switch.

If you have the latest version of fpm in the top of your package (where the fpm.toml file is) you can create a file called "fpm.rsp"
and define aliases for long commands or ones you want to enter frequently. In this example it defines a custom run, build, and test command so if you enter "fpm @build" it would do the same thing as "fpm build -compiler nvfortran -flag "-O3 -cuda -Mpreprocess". you can put multiple ones in there and call them what you want. They do not have to be @run, @test, @build.

@run
option run --compiler nvfortran --flag "-O3 -cuda -Mpreprocess"

@build
option build --compiler nvfortran --flag "-O3 -cuda -Mpreprocess"

@test
option build --compiler nvfortran --flag "-O3 -cuda -Mpreprocess"
```text

 

[jphaupt]

Hi, thanks for the reply! Sure, I made an even simpler example here if you want to take a look: https://github.com/jphaupt/cuda_eg
This example has only .f90 files, but in practice I will want to also use .cuf since the nvidia-provided libraries seem to use it.

I installed my copy of fpm via conda. I happen to be running Manjaro so I also tested the build on the AUR. Both give the same error. I do have gfortran installed, but I'm afraid I don't really understand what you want me to do with it.

[LKedward]

Finally, tangentially related but is there support for using multiple compilers, for different executables? I

Hi @jphaupt, yes you can use fpm with multiple compilers side-by-side. fpm places all build objects in a separate directory based on the chosen compiler and compiler flags - hence you can run different executables produced by different compilers like so:

$ fpm run --compiler mpiifort   # Run executable produced with mpiifort
$ fpm run --compiler nvfortran  # Run executable produced with nvfortran

We can't however specify a different compiler for different source files and so you may have to use preprocessor #ifdefs to switch out the MPI implementation with the CUDA implementation where needed.

[jphaupt]

I have tried the --compiler option with mpifort and nvfortran but both have given me issues (viz the other discussion I posted on the same day about MPI). Anyway, it's probably about time I learn how to use fortran preprocessors. :)

[urbanjost]

fpm assumes it has the source for everything it needs to build by default. You have to tell it if anything is external, so to tell it not to fail when it encounters modules you plan on satisfying from outside the package in the [build] section of the manifest file fpm.toml add a list of the external modules, like the "external-modules" line below:

name = "cuda_eg"
version = "0.1.0"
[build]
auto-executables = true
auto-tests = true
auto-examples = true

external-modules = ["cudafor", "curand", "cublas"]

[install]
library = false

[[ executable ]]
name='ask'
main="main.f90"

Until compiler flags can be specified in the manifest file you will have to use the --flag option. To make that easier
create a file called "fpm.rsp" and with the following lines in it you can enter "fpm @run" and the example should
run. If your fpm command itself is failing all the time with all packages we need to do something about that, but
if you get the
https://github.com/urbanjost/fpm-tools/tree/main/tools/bootstrap/fpm_20210607.F90

and do a "gfortran fpm_20210607.F90 -o fpm" and put that fpm in your path and it gives you trouble too we can build that with some
options that will let us debug what is going on.

To make it a little less tedious to keep typing the flags (which will be able to be specified in the future in the fpm.toml file and/or via an environment variable) you can make a "response" file called fpm.rsp and put lines like this in it

 cat fpm.rsp
################################################################################
@run
option run --compiler nvfortran --flag "-cuda -Mpreprocess"

@build
option build --compiler nvfortran --flag "-cuda -Mpreprocess"

@test
option build --compiler nvfortran --flag "-cuda -Mpreprocess"
################################################################################

and then to build and run your example just enter "fpm @run".

[jphaupt]

The bootstrap build solved all my problems. The fpm.rsp trick along with a preprocessor as suggested by @LKedward will also resolve my last point (as soon as I learn how to use them anyway). This new build also allows me to use mpi using the hacky workaround I mentioned in another thread (and I can just put this workaround in fpm.rsp). Thanks a tonne! I managed to also build with curand successfully. One final question if that's okay... If I have a common library or module(s) for two different projects (e.g. ../common) is there a way for fpm to find and build them or would best practice be to just make a symlink in src? Thanks a lot for developing this software; I really like the idea and I think once I have got a handle of it, it'll become my go-to.

[urbanjost]

Well, it depends on whether the common libraries should be used pre-built or whether you want to build them with the same options as the project each time (especially if ithe code needs pre-processing) and whether you are going to make your package distributable or just need it to work on your machine for your use.

So links are fine, or if the common code is made into it's own package you can list it as a dependency (git can clone local repositories too)
A bash shell script like this shows the idea

 # make two git repositories A and B and initialize them
mkdir test
cd test
fpm new A
(
  cd A
  git add --all
  git commit -m init
)
fpm new B
(
  cd B
  git add --all
  git commit -m init
)
# go into A and make the local git repository B a dependency by adding it to the fpm.toml file
cd A
cat >>fpm.toml <<\EOF
[dependencies]
  B = {  git  =  "/home/urbanjs/venus/V600/github/test/B"}
  #B = {  git  =  "file://home/urbanjs/venus/V600/github/test/B"}
EOF
# change the sample program to use B as well as A
cat >app/main.f90 <<\EOF
program main
  use A, only: A_hello=>say_hello
  use B, only: B_hello=>say_hello
  implicit none
  call A_hello()
  call B_hello()
end program main
EOF
# try it
fpm run

or you can put skeleton code in each project that does an INCLUDE of the actual module code. If you have a bunch of files you can link the directory into your source code tree instead of doing individual files or you can make a link to the directory and use it a local dependency.

If you look at https://github.com/fortran-lang/fpm/blob/master/manifest-reference.md

You can see that you can put code in your source tree that uses INCLUDE and specify the include directory name in the fpm.toml manifest file, link to the directory and use it as a local dependency by putting that in the manifest file if they are just files. If the common files are a pre-built you can specify them as an external module dependency. If they comprise an entire package by themselves you can make them a dependency. If they are on github (as most if not all the examples show) or in your own reposistory server they will then be cloned into your project but not refreshed unless you do an "fpm update" or delete the build/ directory; but as the example script shows (it assumes bash shell in a Linux environment but I think if you have built a package already the basic steps will be clear(?)) you can have a local directory as a dependency too. Depends a bit on how often you are changing the common modules, if you want everything to be self-contained in each package so they are easy to distribute, .... I think the local dependencies in the file you built fpm from require the local dependency directory name to be relative to the root of the package, but that might change to allow full pathnames, although there are pros and cons to that. So the answer is a bit long because "it depends". I am thinking we need more examples in the documentation. As mentioned in your other thread there are changes coming to make MPI and OpenMP usage easier but nvfortran can take a LOT of special options that do not quite fit the basic model of building all your source from a single package, which fpm supports/endorses where possible. I am thinking your cases would make great examples and safe the next guy a lot of effort.

[urbanjost]

Forgot showing and example with curand last time, forgot to mention to @awvwgk the problem with the conda version. He mighthave seen that. @awvwgk? Might want to post a short curand example here. I have had trouble getting nvfortran to work with some codes, but when it does it can be fantastic. I have an old code that can run for days that I got a 13x speed-up with less than a man-day's effort. That felt good.