MPI parallelization of thermodynamics

[draenog]

Background

This PR is part of #120, where the basis of strategy for MPI parallelisation is described.

Change description

As all thermodynamics operations are local to a grid cell all required MPI communication, is handled by NetCDF4 library. Therefore the only required steps are:

• Initialize and finalize MPI stack
• Read decomposition metadata on each rank
• Read and write the necessary part of the grid on each rank (depends on Issue264 netcdf strings #330)
• Tests for parallel I/O

NOTE PR #432 should be merged into this branch before it is merged into develop

To run run_simple_example.sh you will need to generate the following netcdf file

$ ncdump partition.nc 
netcdf partition {
dimensions:
        P = 1 ;
        L = 1 ;
        globalX = 30 ;
        globalY = 30 ;

group: bounding_boxes {
  variables:
        int global_x(P) ;
        int global_y(P) ;
        int local_extent_x(P) ;
        int local_extent_y(P) ;
  data:

   global_x = 0 ;

   global_y = 0 ;

   local_extent_x = 30 ;

   local_extent_y = 30 ;
  } // group bounding_boxes
}

[dorchard]

develop branch now in a position for this to be turned into a non-draft PR, once the conflicts are resolved, which @draenog will do next week.

[TomMelt]

Based on discussion with Tim and Einar, this should be rebased onto feat_mar3. I will then look to rebase #265 onto this (mpi_local) branch.

[einola]

We need to be careful not to use MPP_WORLD as the communicator because the model should receive the communicator from OASIS in a coupled mode.

[draenog]

We need to be careful not to use MPP_WORLD as the communicator because the model should receive the communicator from OASIS in a coupled mode.

Yes, it is a good point. I think it is currently quite well encapsulated. Constructor of Nextsim:Model takes MPI communicator as an argument which is then saved in ModelMetadata.

[timspainNERSC]

• Need to check that the changes to DevGrid go through cleanly now that it doesn't exist in develop.
• Need to check DimensionSetter() is still correct in the MPI version now that the ordering of the array dimensions has been swapped.

Otherwise, this looks like a god set of MPI changes.