Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

XIOS Integration (MVP) #265

Closed
wants to merge 57 commits into from
Closed

XIOS Integration (MVP) #265

wants to merge 57 commits into from

Conversation

a-smith-github
Copy link
Contributor

@a-smith-github a-smith-github commented Apr 6, 2023

Nextsim XIOS Integration

Closes #218

This is a master pull request tracking the integration of XIOS for File I/O into Nextsim DG. Some of the sub-components are being tracked by separate PRs. See the design document on the Wiki for more information on XIOS Integration.

The core functionality of XIOS is included as part of this PR as the other two components will require it.

Core XIOS functionality

Closes #293
XIOS uses a client-server model. To use XIOS we must first initialize the clients on each process (cxios_init_client) and subsequently initialize the server (cxios_context_initialize). Here are the necessary steps to be able to setup the XIOS client & server processes.

  • add XIOS to the cmake build process i.e., CMakeLists.txt
  • initialize client and server process from inside nextsimdg
  • verify correct setup and exit
  • add unit tests
  • merge in Kacper's latest MPI branch (mpi_local) (I am dependent on doctest with MPI support)
  • rewrite XIOS tests to use mpi-aware doctest

NOTE currently addressing bug in Xios class that affects the getCalendarTimestep method. For now I have implemented a work around (I have now fixed this with #424 but I want to find a better solution for mpi-aware doctests).

Once the essentials have been created to begin using XIOS my intention is for the development branches to be structured as follows:

Merge instructions

When development of all XIOS features is ready (#422, #421 and this PR). The plan is to merge them (in reverse order) into this branch, finally merging this branch back into feat_mar3

Notes on XIOS

  • XIOS is a third party library which was designed to handle file I/O asynchronously
  • Documentation is possibly out-dated (last update 2018) and some of the interface methods are undocumented which adds complexity.
  • You can read more in the UserGuide and the ReferenceGuide
  • There is a more recent training session (2020).
  • XIOS requires a configuration .xml file defining a calendar system to run
  • Most settings can be either set in the .xml or at runtime (see UserGuide for more info)

@a-smith-github a-smith-github self-assigned this Apr 6, 2023
@a-smith-github a-smith-github marked this pull request as draft April 6, 2023 13:40
@a-smith-github a-smith-github added the ICCS Tasks or reviews for the ICCS team label Apr 6, 2023
@a-smith-github a-smith-github linked an issue Apr 27, 2023 that may be closed by this pull request
@mondus
Copy link

mondus commented May 11, 2023

CMake progress is good but needs some tidying. E.g. Proof of concept build is working.

a-smith-github and others added 17 commits October 4, 2023 17:14
nextsim build fails due to incorrect order of libraries

-lxios needs to come before -lnetcdf

also

for gnu compiler we need -lgfortran and for intel compiler we need
-lifcore

TODO:

Findxios in conjuction with the CMakeLists.txt needs fixing still so
that it can work when xios is not installed.
I have manually changed lib ifcore to lgfortran. This will need to fixed
properly for intel and gnu compiler and not hard-coded.

this is a temporary fix.

The order of libs is also incorrect for XIOS_LIBRARY_LIST. xios depends
on netcdf so it should come second.

There was an erroneous ::Xios for the constructor when creating
xios_handler.
I have updated the ubuntu install and setup for XIOS and nextsimdg

I have removed unnecessary packages and hopefully xios will now build.
@TomMelt TomMelt assigned TomMelt and unassigned a-smith-github Oct 4, 2023
@TomMelt TomMelt changed the title WIP: XIOS Integration (MVP) XIOS Integration (MVP) Oct 4, 2023
This was referenced Oct 5, 2023
I have updated and simplified the Xios class

We have methods to get/set common variables:
* Calendar origin
* Calendar start
* Calendar timestep

There are some initialization method and an isInitialized function to
verify.

I have simplified the string conversion string for datetime
@TomMelt TomMelt added this to the 3 Stand-alone model milestone Oct 16, 2023
@TomMelt
Copy link
Contributor

TomMelt commented Feb 12, 2024

this PR has got stale due to significant changes since the beginning of XIOS project. This PR will be superseded by #496 .

@TomMelt TomMelt closed this Feb 12, 2024
@TomMelt TomMelt deleted the feature218_xios_io branch February 12, 2024 09:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ICCS Tasks or reviews for the ICCS team
Projects
Development

Successfully merging this pull request may close these issues.

Add XIOS IO Capability
3 participants