Skip to content

Commit

Permalink
Update documentation for preCICE v3 / latest state of the adapter (#39)
Browse files Browse the repository at this point in the history
  • Loading branch information
@MakisH
MakisH authored Apr 3, 2024
1 parent 76484f0 commit 83ca324
Show file tree
Hide file tree
Showing 4 changed files with 165 additions and 212 deletions.
175 changes: 3 additions & 172 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,177 +1,8 @@
# SU2-preCICE adapter

The SU2 adapter now leverages the SU2 Python wrapper and preCICE Python bindings to couple SU2 using preCICE in a minimally invasive way. This adapter simply updates existing functions and implements new ones in the SU2 Python wrapper that allow for simple preCICE implementation with implicit coupling. In addition, Python scripts are provided that can be used to quickly run FSI or CHT, or can serve as templates for your own custom application. This adapter currently works for SU2 versions 7.5.0 and 7.5.1 "Blackbird". Both conjugate heat-transfer and fluid-structure interaction can be completed with this adapter.
This is a [preCICE](https://precice.org/) adapter for [SU2](https://su2code.github.io/) versions 7.5.0 and 7.5.1 "Blackbird",
including examples for conjugate heat transfer and fluid-structure interation simulations.

This adapter has been designed to work when using the compressible solver for unsteady problems with dual-time stepping, for single-zone problems. Implicit coupling currently saves the flow solution, turbulence solution, and the mesh solution (for mesh deformation). Species transport and transition model variables at this time are not saved, but may be straightforward to implement.
Read more in the [documentation](https://precice.org/adapter-su2-overview.html).

**Note:** In its current state, the SU2 adapter is using the Python wrapper of SU2. The [previous implementation](https://github.com/precice/su2-adapter/tree/ab843878c1d43302a4f0c66e25dcb364b7787478) was directly editing the C++ source files of SU2. There is also a [version relying on the Python wrapper that however works with preCICE v2](https://github.com/precice/su2-adapter/commit/a87a1ed57e14dca97f1e47aab44632a254714004).

<!-- tocstop -->
## Building the adapter

The adapter depends on [SU2](https://su2code.github.io/), [preCICE v3](https://precice.org/installation-overview.html), and the [preCICE Python bindings](https://precice.org/installation-bindings-python.html).

The script `su2AdapterInstall` replaces a few files in the SU2 source code. You then need to build SU2 from source, install it into a prefix (`SU2_RUN`) and add that to your `PATH`.

To run SU2, you can use the provided Python scripts `SU2_preCICE_CHT.py` and `SU2_preCICE_FSI.py`, which call SU2 via its Python interface.

### Get the SU2 source

Download SU2 v7.5.1 "Blackbird" source directly from the [SU2 releases on GitHub](https://github.com/su2code/SU2/releases/tag/v7.5.1). Note that both swig and mpi4py must be installed to use the SU2 Python wrapper, which needs to be enabled using the flag `-Denable-pywrapper=true` when building SU2.

### Build the adapter

In order to couple SU2 using preCICE, `python_wrapper_structure.cpp` and `CDriver.hpp` must be updated. This adapter provides the updated files. The shell script `su2AdapterInstall`, which comes with this adapter, automatically replaces the files in your SU2 directory with these updated files and provides the correct commands to re-configure and re-install SU2 with the added adjustments. This script and the build/install procedure rely on a few environment variables. Set the `SU2_HOME` to your SU2 source directory, the `SU2_RUN` to a directory where SU2 will install executable files, and add `SU2_RUN` to your `PATH`, and `PYTHONPATH` variables. For example, SU2 will show this message:

```text
Based on the input to this configuration, add these lines to your .bashrc file:
export SU2_RUN=/home/myuser/software/SU2_RUN/bin
export SU2_HOME=/home/myuser/software/SU2-7.5.1
export PATH=$PATH:$SU2_RUN
export PYTHONPATH=$PYTHONPATH:$SU2_RUN
Use './ninja -C build install' to compile and install SU2
```

which means you should set:

```shell
export SU2_RUN="/home/myuser/software/SU2_RUN"
export SU2_HOME="/home/myuser/software/SU2-7.5.1"
export PATH="${SU2_RUN}/bin:/path/to/su2-adapter/run/:$PATH"
export PYTHONPATH="${SU2_RUN}/bin:${PYTHONPATH}"
```

In particular, make sure that `SU2_RUN` points to a directory into which you have write access. You later will need to pass this to the SU2 build system (meson) with `--prefix`.

To copy the adapter files into `SU2_HOME`, run:

```shell
./su2AdapterInstall
```

The installation script will prompt you to follow commands. Check the output and follow the commands. For example:

```shell
./meson.py build -Denable-pywrapper=true --prefix=$SU2_RUN
./ninja -C build install
```

This will trigger the normal building procedure of SU2. Please refer to the installation instructions of SU2 for more details. SU2 should be built with MPI support in order to make use of parallel functionalities and must be built with pywrapper functionality enabled.

To be able to run the FSI and CHT Python scripts included in the adapter from anywhere, add to your `~/.bashrc`:

```shell
export PATH=/path/to/adapter/run:$PATH
```

## Running simulations

After successfully installing the adapted SU2, use the provided FSI/CHT scripts to run simulations, or build your own. Note that these scripts currently are designed for a single coupling mesh, called `interface`. However, it is extremely easy to update these scripts to handle a different BC name and/or multiple interfaces. They are provided simply for their ease of use.

### Fluid-structure interaction

#### SU2 configuration file for FSI

To set up a single-interface FSI problem for coupling with preCICE, the SU2 config file should have the following:

```text
DEFORM_MESH= YES
MARKER_DEFORM_MESH= ( interface )
```

The `interface` marker should also be set as a wall boundary, such as `MARKER_EULER` or `MARKER_ISOTHERMAL`.

#### Running an FSI simulation

By default, in the `SU2_preCICE_FSI.py` script, the following settings are automatically used for coupling with preCICE:

- preCICE Participant Name: `Fluid`
- preCICE Config File: `../precice-config.xml`
- preCICE Mesh Name: `Fluid-Mesh`
- preCICE Read Data: `Displacements`
- preCICE Write Data: `Forces`

To run with these settings:

```shell
SU2_preCICE_FSI.py -f SU2_config_file.cfg --parallel
```

The `--parallel` flag must **always** be used when SU2 is built in parallel, even if running on a single process. If you do not build SU2 with MPI, do not include it.

The read/write data variables are hardcoded, but the participant name, config file, and mesh name can be changed using flags in the call to the Python file. In general, to run an FSI case:

```shell
SU2_preCICE_FSI.py -f SU2_config_file.cfg -p participant_name -c precice_config_file -m precice_mesh_name --parallel
```

### Conjugate heat transfer

#### SU2 configuration file for CHT

To set up a single-interface CHT problem for coupling with preCICE, the SU2 config file should have the following:

```text
% For having SU2 read temperature, write heat flux:
MARKER_ISOTHERMAL= (interface, ______)
%
% For having SU2 read heat flux, write temperature (the -r flag):
MARKER_HEATFLUX= (interface, ______)
%
% And in both cases include:
MARKER_PYTHON_CUSTOM= (interface)
```

Note that the blank spots in the isothermal and heat flux markers are the initial BC values. If there is a data initialization from another solver, they will be updated and are not important.

#### Running a CHT simulation

By default in the `SU2_preCICE_CHT.py` script, the following settings are automatically used for coupling with preCICE:

- preCICE Participant Name: `Fluid`
- preCICE Config File: `../precice-config.xml`
- preCICE Mesh Name: `Fluid-Mesh`
- preCICE Read Data: `Temperature`
- preCICE Write Data: `Heat-Flux`

To run with these settings:

```shell
SU2_preCICE_CHT.py -f SU2_config_file.cfg --parallel
```

The `--parallel` flag must **always** be used when SU2 is built in parallel, even if running on a single process. If you do not build SU2 with MPI, do not include it.

The read/write data for CHT can be reversed if the preCICE config file specifies for the fluid to read heat flux and write temperature. This can easily be accomplished with the `-r` flag:

```shell
SU2_preCICE_CHT.py -f SU2_config_file.cfg -r --parallel
```

The participant name, config file, and mesh name can be changed using flags in the call to the Python file. In general, to run a CHT case:

```shell
SU2_preCICE_CHT.py -f SU2_config_file.cfg -p participant_name -c precice_config_file -m precice_mesh_name --parallel
```

### Running in parallel

The Python scripts can very easily be run in parallel by just pre-pending the Python script call like:

```shell
mpirun -n 8 python3 SU2_preCICE_CHT.py -f SU2_config_file.cfg --parallel
```

**NOTE**: As of SU2 v7.5.1: Deforming `MARKER_EULER`'s are buggy when simulations are run in parallel, leading to unexpected results. More information can be found at this discussion here: https://github.com/su2code/SU2/discussions/1931.

## Important note on restarts

This code **has not been tested** for restarts using initializations *from* SU2. Any restarted simulations should have SU2 be the first participant and receive initialization data. It is possible that, if SU2 must send initialization data, that it is incorrect (it may use default values in the config file, or just be zeros if the data hasn't been computed until after/during a first iteration). Admittedly, this is from a lack of understanding of the specifics of how SU2 operates and there may not be a trivial work-around.

## Further notes

Result files (vtu) generated from SU2 might be incompatible with your ParaView version. For example, ParaView 5.11.2 on Ubuntu 22.04 is known to fail with SU2 7.5.1 result files, but ParaView 5.12 works.

The replacement files included in this repository might be long, but they only introduce minimal changes compared to the original SU2 code (mainly related to checkpointing for implicit coupling). When updating to newer SU2 versions, compare the bundled and the old unmodified files in a diff tool, and start by copying the same changes into the new source. See the `replacement_files/README.md` for more details.
122 changes: 100 additions & 22 deletions docs/configure.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,34 +5,112 @@ keywords: adapter, su2, development, modules
summary: "Modify SU2 configuration file, specify interfaces by SU2 markers, run SU2 either serial or parallel"
---

## SU2 configuration file
After successfully installing the adapted SU2, use the provided FSI/CHT scripts to run simulations, or build your own. Note that these scripts currently are designed for a single coupling mesh, called `interface`. However, it is extremely easy to update these scripts to handle a different BC name and/or multiple interfaces. They are provided simply for their ease of use.

The adapter is turned on or off via the native SU2 configuration file. If it is turned off, SU2 executes in its original version. Moreover, the adapter is configured in the SU2 configuration file. The following adapter-related options are currently available (default values given in brackets):
## Fluid-structure interaction

1. `PRECICE_USAGE` (NO): Determines whether a preCICE-coupled simulation is run or not.
2. `PRECICE_VERBOSITYLEVEL_HIGH` (NO): Produces more output, mainly for debugging purposes.
3. `PRECICE_LOADRAMPING` (NO): Allows to linearly ramp up the load on the structural component at the beginning of the simulation. This may resolve stability issues due to large loads at the beginning of simulations.
4. `PRECICE_CONFIG_FILENAME` (precice-config.xml): Location and name of the preCICE configuration file.
5. `PRECICE_PARTICIPANT_NAME` (SU2): Name of the participant in the preCICE configuration file.
6. `PRECICE_MESH_NAME` (SU2-Mesh): Name of the mesh in the preCICE configuration file.
7. `PRECICE_READ_DATA_NAME` (Displacements): Name of the read data in the preCICE configuration file. The SU2-adapter supports reading absolute displacements and relative displacements. The adapter picks up the respective data field according to the name: 'Displacement' for absolute displacement and 'DisplacementDelta' for relative displacement. {% important %}
Note that reading 'Displacement' data has been added for compatibility reasons with our tutorials and it cannot be used with `extrapolation` in your configuration. Only the relative `DisplacementDelta` data supports this feature.
{% endimportant %}
8. `PRECICE_WRITE_DATA_NAME` (Forces): Name of the write data in the preCICE configuration file.
9. `PRECICE_WETSURFACE_MARKER_NAME` (wetSurface): Name of the marker, which identifies the FSI interface in the geometry file of SU2.
10. `PRECICE_LOADRAMPING_DURATION` (1): Number of time steps, in which the load ramping is active, counting from the beginning of the simulation. The ramped load increases linearly with each time step.
11. `PRECICE_NUMBER_OF_WETSURFACES` (1): In case multiple FSI-interfaces exist, their count needs to specified here.
### SU2 configuration file for FSI

If multiple interfaces exist, the names of all related entries (`PRECICE_WETSURFACE_MARKER_NAME`, `PRECICE_READ_DATA_NAME`,`PRECICE_WRITE_DATA_NAME`, `PRECICE_MESH_NAME`) must be appended by consecutive numbers. Hence, the names (also in the geometry file) need to be alike differing only by the appending number, which must be successively increasing from zero. E.g. for three interfaces, the marker name could be defined as `PRECICE_WETSURFACE_MARKER_NAME= wetSurface` in the SU2 configuration file, while the markers in the geometry file would need to be named *wetSurface*, *wetSurface1* and *wetSurface2*.
To set up a single-interface FSI problem for coupling with preCICE, the SU2 config file should have the following:

Moreover, in the SU2 configuration file grid movement must be allowed: `GRID_MOVEMENT= YES` and the type of grid movement must be set correctly for preCICE-coupled simulations: `GRID_MOVEMENT_KIND= PRECICE_MOVEMENT`. Also, the boundary, which is allowed to move needs to be specified. Here the name of the FSI-interface marker including its appending identifying number as stated above needs to be used, e.g., `MARKER_MOVING= ( wetSurface0 )`. If multiple FSI-interfaces exist (as in the example above), this may look like `MARKER_MOVING= ( wetSurface, wetSurface1, wetSurface2 )`.
```text
DEFORM_MESH= YES
MARKER_DEFORM_MESH= ( interface )
```

## Running the adapted SU2 executable
The `interface` marker should also be set as a wall boundary, such as `MARKER_EULER` or `MARKER_ISOTHERMAL`.

Since the adapter (as well as its options) is turned on or off via the SU2 configuration file, the execution procedure is just as for the original version of SU2. For execution with one process working on the fluid domain from the directory, in which both the SU2_CFD executable and the SU2 configuration file are located:
### Running an FSI simulation

`./SU2_CFD su2ConfigurationFile.cfg`
By default, in the `SU2_preCICE_FSI.py` script, the following settings are automatically used for coupling with preCICE:

The adapter is designed such that it can be executed in an intra-parallel manner meaning that the flow domain is decomposed into several parts. The execution is then as follows (again assuming that executable and configuration file are within the current directory; exemplifying a decomposition of the fluid domain with eight processes):
- preCICE Participant Name: `Fluid`
- preCICE Config File: `../precice-config.xml`
- preCICE Mesh Name: `Fluid-Mesh`
- preCICE Read Data: `Displacements`
- preCICE Write Data: `Forces`

`mpirun -n 8 ./SU2_CFD su2ConfigurationFile.cfg`
To run with these settings:

```shell
SU2_preCICE_FSI.py -f SU2_config_file.cfg --parallel
```

The `--parallel` flag must **always** be used when SU2 is built in parallel, even if running on a single process. If you do not build SU2 with MPI, do not include it.

The read/write data variables are hardcoded, but the participant name, config file, and mesh name can be changed using flags in the call to the Python file. In general, to run an FSI case:

```shell
SU2_preCICE_FSI.py -f SU2_config_file.cfg -p participant_name -c precice_config_file -m precice_mesh_name --parallel
```

## Conjugate heat transfer

### SU2 configuration file for CHT

To set up a single-interface CHT problem for coupling with preCICE, the SU2 config file should have the following:

```text
% For having SU2 read temperature, write heat flux:
MARKER_ISOTHERMAL= (interface, ______)
%
% For having SU2 read heat flux, write temperature (the -r flag):
MARKER_HEATFLUX= (interface, ______)
%
% And in both cases include:
MARKER_PYTHON_CUSTOM= (interface)
```

Note that the blank spots in the isothermal and heat flux markers are the initial BC values. If there is a data initialization from another solver, they will be updated and are not important.

### Running a CHT simulation

By default in the `SU2_preCICE_CHT.py` script, the following settings are automatically used for coupling with preCICE:

- preCICE Participant Name: `Fluid`
- preCICE Config File: `../precice-config.xml`
- preCICE Mesh Name: `Fluid-Mesh`
- preCICE Read Data: `Temperature`
- preCICE Write Data: `Heat-Flux`

To run with these settings:

```shell
SU2_preCICE_CHT.py -f SU2_config_file.cfg --parallel
```

The `--parallel` flag must **always** be used when SU2 is built in parallel, even if running on a single process. If you do not build SU2 with MPI, do not include it.

The read/write data for CHT can be reversed if the preCICE config file specifies for the fluid to read heat flux and write temperature. This can easily be accomplished with the `-r` flag:

```shell
SU2_preCICE_CHT.py -f SU2_config_file.cfg -r --parallel
```

The participant name, config file, and mesh name can be changed using flags in the call to the Python file. In general, to run a CHT case:

```shell
SU2_preCICE_CHT.py -f SU2_config_file.cfg -p participant_name -c precice_config_file -m precice_mesh_name --parallel
```

## Running in parallel

The Python scripts can very easily be run in parallel by just pre-pending the Python script call like:

```shell
mpirun -n 8 python3 SU2_preCICE_CHT.py -f SU2_config_file.cfg --parallel
```

{% note %}
As of SU2 v7.5.1: Deforming `MARKER_EULER`'s are buggy when simulations are run in parallel, leading to unexpected results. More information can be found at [this SU2 discussion](https://github.com/su2code/SU2/discussions/1931).
{% endnote %}

## Important note on restarts

This code **has not been tested** for restarts using initializations *from* SU2. Any restarted simulations should have SU2 be the first participant and receive initialization data. It is possible that, if SU2 must send initialization data, that it is incorrect (it may use default values in the config file, or just be zeros if the data hasn't been computed until after/during a first iteration). Admittedly, this is from a lack of understanding of the specifics of how SU2 operates and there may not be a trivial work-around.

## Further notes

Result files (vtu) generated from SU2 might be incompatible with your ParaView version. For example, ParaView 5.11.2 on Ubuntu 22.04 is known to fail with SU2 7.5.1 result files, but ParaView 5.12 works.

The replacement files included in this repository might be long, but they only introduce minimal changes compared to the original SU2 code (mainly related to checkpointing for implicit coupling). When updating to newer SU2 versions, compare the bundled and the old unmodified files in a diff tool, and start by copying the same changes into the new source. See the `replacement_files/README.md` for more details.
Loading

0 comments on commit 83ca324

Please sign in to comment.