Usage

Convenience Makefile

The Makefile in the repository holds a number of convenience targets and its use is entirely optional. It creates a build subdirectory and runs cmake to configure the project; the executable ends up in build/run.

Target	Effect
`make debug`	Switch to a debug build and compile
`make release`	Switch to a release build and compile
`make edit_cache`	Run `ccmake` in the build directory

Executable and runtime parameter files

After compiling you end up with an executable at build/run/ryujin. It takes a parameter file location as its single optional argument. Without an argument it looks for ryujin.prm in the working directory of the executable, that is build/run:

cd build/run
./ryujin                      # uses ryujin.prm
./ryujin my_parameter_file.prm

Example parameter files live in the prm subdirectory.

prm/verification contains parameter files and baseline output vectors documenting expected errors and convergence rates for various analytical solutions. Each configuration compares the simulation result at final time against a known analytic solution and records the normalized L1, L2 and L_∞ error norms, summed over all components. Run all of these in double precision.

prm/benchmarks contains parameter files for well known benchmark configurations. These typically have no analytical solution, but their solution structure is well known, so they are compared in the eyeball norm. To run the Mach 3 cylinder with 2.36M gridpoints on a machine with 16 cores and two threads per core:

cd build/run
DEAL_II_NUM_THREADS=2 mpirun -np 16 ryujin prm/benchmarks/euler-mach3-cylinder-2d.prm

To obtain a full list of supported runtime parameters and their default values, invoke ./ryujin without a ryujin.prm in the path:

% ./ryujin
[INFO] initiating flux capacitor
[INFO] Default parameter file »ryujin.prm« not found.
[INFO] Creating template parameter files...
% ls
[...]
default_parameters-euler-2d-description.prm
default_parameters-navier_stokes-2d-description.prm
default_parameters-shallow_water-2d-description.prm
default_parameters-euler_aeos-2d-description.prm
default_parameters-scalar_conservation-2d-description.prm

These generated files carry detailed annotated configuration options. Their 1d and 3d counterparts are generated alongside them.

Equation modules

The equation is selected at run time through the equation parameter. The following modules are available.

euler: the compressible Euler equations with a polytropic equation of state.
euler barotropic: the compressible Euler equations with a barotropic equation of state, such as the isentropic or isothermal one.
euler aeos: the compressible Euler equations with an arbitrary or tabulated equation of state.
navier stokes: the compressible Navier-Stokes equations with a polytropic equation of state, Newtonian fluid model, and Fourier’s law for the heat flux.
euler poisson, euler poisson barotropic, euler poisson aeos: a charged single-species fluid governed by the compressible Euler equations with the respective equation of state, subject to an electrostatic force model with a background magnetic field.
shallow water: the shallow water equations.
scalar conservation: scalar conservation equations with a user-supplied flux, a greedy wave-speed estimate to maintain an invariant domain, an entropy-viscosity commutator indicator with a general entropy-like function, and a customizable convex limiter.

Output file format

ryujin writes temporal snapshots in .vtu format, which you can open and inspect with ParaView, available through your package manager. In principle ryujin can be modified to write any format deal.II supports; see source/vtu_output.template.h.

There is also rudimentary support for writing instantaneous, time averaged, or space integrated primitive values, and their second moments, on user-defined level sets.

Controlling parallelism and screen output

ryujin uses MPI and thread parallelization. The degree of MPI parallelism is controlled by the MPI launcher; the number of threads created per rank is controlled by the DEAL_II_NUM_THREADS environment variable. To run on 8 ranks with 4 threads each:

DEAL_II_NUM_THREADS=4 mpirun -np 8 ./ryujin

Compile-time options

Compile-time options are set on the command line during configuration with cmake, or interactively with ccmake:

cd build
ccmake .

The convenience Makefile invokes ccmake from anywhere in the source tree:

make edit_cache

The most important option is CMAKE_BUILD_TYPE, which switches between a debug (Debug) and a release (Release) configuration. The convenience Makefile sets this directly with make debug and make release from the top level directory.

Changing the other options is rarely needed, and when in doubt they are best left at their defaults. For completeness:

Option	Effect	Default
`CMAKE_BUILD_TYPE`	Build in `Release` or `Debug` mode
`NUMBER`	`double` for double precision, `float` for single	`double`
`ASYNC_MPI_EXCHANGE`	Asynchronous “communication hiding” MPI exchange	`OFF`
`DEBUG_EXPENSIVE_BOUNDS_CHECK`	Debug code paths for additional limiter bounds checks	`OFF`
`DEBUG_OUTPUT`	Debug output	`OFF`
`DEBUG_SANITIZER`	Address and UBSAN sanitizers for a debug build	`OFF`
`DEBUG_SYMMETRY_CHECK`	Debug code paths verifying `d_ij`, `c_ij`, `m_ij` for (anti)symmetry	`OFF`
`DENORMALS_ARE_ZERO`	Disable floating point denormals	`ON`
`WITH_DOXYGEN`	Support for doxygen, and build the documentation
`WITH_EOSPAC`	Support for the EOSPAC6/Sesame tabulated equation of state database	autodetect
`WITH_LIKWID`	Support for Likwid stethoscope mode, for Intel performance counters	`OFF`
`WITH_NVTX`	NVTX instrumentation of timer sections for the CUDA profiler (`nsys`, `ncu`)	`OFF`
`WITH_OPENMP`	Multithreading via OpenMP	autodetect