Skip to content

Installation: Simulation Options

Hsi-Yu Schive edited this page May 22, 2024 · 36 revisions

Format

All compile-time simulation options in the Makefile are in the following two formats:

SIMU_OPTION += -DOPTION1
SIMU_OPTION += -DOPTION2=OPTION2_ADOPTED

which will enable OPTION1 and assign OPTION2_ADOPTED to OPTION2. For example, to (i) enable gravity and (ii) adopt the CTU fluid scheme, set

SIMU_OPTION += -DGRAVITY
SIMU_OPTION += -DFLU_SCHEME=CTU

To disable an option, just comment it out with #. For example, to disable gravity, use

#SIMU_OPTION += -DGRAVITY

Caution: option values (if any) must be set explicitly since there are no default values. For example, SIMU_OPTION += -DFLU_SCHEME without assigning any value to the option FLU_SCHEME is invalid. In addition, do not insert any space before and after the equal sign =. For example, use -DFLU_SCHEME=CTU instead of -DFLU_SCHEME = CTU.

Option List

All compile-time simulation options are listed below. They are classified into the following categories:

Caution: some combinations are mandatory (e.g., RSOLVER must be set when FLU_SCHEME=CTU), while some combinations are prohibited (e.g., PARTICLE is not supported when both GRAVITY and TRACER are disabled). See the "Restriction" of each option carefully.

Physical Modules

Option Value Description Restriction
MODEL HYDRO
ELBDM
Physical models, where ELBDM is for ψDM Must be set in any cases; ELBDM is not released yet
GRAVITY Enable gravity Must enable SUPPORT_FFTW; may need to set FFTW2/3_PATH
PARTICLE Enable particles Must enable GRAVITY or TRACER
SUPPORT_GRACKLE Enable GRACKLE May need to set GRACKLE_PATH; only support EOS=EOS_GAMMA/EOS_COSMIC_RAY; doesn't support COMOVING

Hydro Options

-- see Hydro for the related runtime parameters and other settings

Option Value Description Restriction
FLU_SCHEME RTVD
MHM
MHM_RP
CTU
Hydro schemes. RTVD: relaxing TVD; MHM: MUSCL-Hancock; MHM_RP: VL scheme; CTU: corner transport upwind MHD only supports MHM, MHM_RP, and CTU; SRHD only supports MHM and MHM_RP; COSMIC_RAY only supports MHM_RP
LR_SCHEME PLM
PPM
Spatial reconstruction. PLM: piecewise linear; PPM: piecewise parabolic Useless for FLU_SCHEME=RTVD
RSOLVER EXACT
ROE
HLLE
HLLC
HLLD
Riemann solvers Useless for FLU_SCHEME=RTVD; EXACT is experimental; pure hydrodynamics supports EXACT/ROE/HLLE/HLLC; MHD supports ROE/HLLE/HLLD; SRHD and COSMIC_RAY support HLLE/HLLC
DUAL_ENERGY DE_ENPY Enable dual energy formalism Not supported for FLU_SCHEME=RTVD
NCOMP_PASSIVE_USER ≥ 0 Number of user-defined passive scalars See here for details; not supported for FLU_SCHEME=RTVD
MHD Magnetohydrodynamics
SRHD Special relativistic hydrodynamics Must adopt EOS=EOS_TAUBMATHEWS
COSMIC_RAY Cosmic rays Must adopt EOS=EOS_COSMIC_RAY
CR_DIFFUSION Cosmic-ray diffusion Must enable both COSMIC_RAY and MHD
EOS EOS_GAMMA
EOS_ISOTHERMAL
EOS_COSMIC_RAY
EOS_TAUBMATHEWS
EOS_USER
Equation of state The following options only support EOS_GAMMA: FLU_SCHEME=RTVD/CTU, RSOLVER=EXACT/ROE, COMOVING, DUAL_ENERGY; see also BAROTROPIC_EOS
BAROTROPIC_EOS Is EOS barotropic? Must be disabled for EOS_GAMMA and enabled for EOS_ISOTHERMAL

Gravity Options

-- see Gravity for the related runtime parameters and other settings

Option Value Description Restriction
POT_SCHEME SOR
MG
Poisson solver. SOR: successive-overrelaxation (recommended); MG: multigrid Must be set when GRAVITY is enabled
STORE_POT_GHOST Store the ghost-zone potential (recommended when PARTICLE is enabled) Must be enabled when both STAR_FORMATION and STORE_PAR_ACC are adopted
UNSPLIT_GRAVITY Use operator-unsplit method to couple gravity to the adopted physical model (recommended) Not supported for MODEL=ELBDM
COMOVING Cosmological simulations

Particle Options

-- see Particles for the related runtime parameters and other settings

Option Value Description Restriction
TRACER Enable tracer particles
STORE_PAR_ACC Store particle acceleration (recommended)
STAR_FORMATION Enable star formation Must enable STORE_POT_GHOST when using STORE_PAR_ACC
FEEDBACK Enable feedback from particles to grids (and vice versa) see here for details
PAR_NATT_USER ≥ 0 Number of user-defined particle attributes See here for details

In Situ Python Analysis Options

-- see In Situ Python Analysis for the related runtime parameters and other settings

Option Value Description Restriction
SUPPORT_LIBYT Enable libyt for in situ Python analysis May need to set LIBYT_PATH
LIBYT_USE_PATCH_GROUP Use patch groups instead of patches as the grid unit for better performance (recommended) Must enable SUPPORT_LIBYT
LIBYT_INTERACTIVE Activate interactive Python prompt in in situ analysis Must enable SUPPORT_LIBYT and compile libyt in interactive mode

Miscellaneous Options

-- AMR, GPU, parallelization, optimizations, ...

Option Value Description Restriction
NLEVEL ≥ 1 Maximum number of AMR levels including the root level. Do not confuse with the MAX_LEVEL runtime parameter.
MAX_PATCH ≥ 8 Maximum number of patches that can be allocated on each AMR level (recommended value: 1000000 or even larger since it is not memory-consuming)
PATCH_SIZE ≥ 8 Number of cells along each direction in a single patch Must be an even number
GPU Enable GPU acceleration Must specify GPU_COMPUTE_CAPABILITY as well; may need to set CUDA_PATH
GPU_COMPUTE_CAPABILITY Three digits GPU Compute Capability (e.g., 890 for GeForce RTX 4090) Must enable GPU
GAMER_DEBUG Run GAMER in a debug mode
BITWISE_REPRODUCIBILITY Enable bitwise reproducibility. It may deteriorate performance, especially for runs with a large number of particles.
TIMING Record the wall time of various GAMER routines in the file Record__Timing (recommended)
TIMING_SOLVER Record the wall time of individual GPU solvers in the file Record__Timing. It will disable the CPU/GPU overlapping and thus deteriorate performance notably. Must enable TIMING
FLOAT8 Enable double precision floating-point accuracy for grid fields. Note that it could have a serious impact on GPU performance.
FLOAT8_PAR Enable double precision floating-point accuracy for particles. It will be set to FLOAT8 by default.
SERIAL Run GAMER in a serial mode (i.e., no MPI; but OpenMP is still supported) Must disable LOAD_BALANCE
LOAD_BALANCE HILBERT Enable load balancing using a space-filling curve (see MPI and OpenMP) Must disable SERIAL; may need to set MPI_PATH
OPENMP Enable OpenMP (see MPI and OpenMP) Must set the compilation flag OPENMPFLAG
SUPPORT_HDF5 Enable HDF5 output (see Outputs) May need to set HDF5_PATH
SUPPORT_GSL Enable GNU scientific library May need to set GSL_PATH
SUPPORT_FFTW FFTW2
FFTW3
Enable FFTW May need to set FFTW2/3_PATH
RANDOM_NUMBER RNG_GNU_EXT
RNG_CPP11
Random number generators. RNG_GNU_EXT: GNU extension drand48_r; RNG_CPP11: c++11 <random> Use RNG_GNU_EXT for compilers supporting GNU extensions (they may not be supported on macOS); use RNG_CPP11 for compilers supporting c++11 (one may need to add -std=c++11 to CXXFLAG → see Compiler and Flags)

Links

Clone this wiki locally