GPU support

Classes
class	ryujin::MirroredStorage< Derived >
class	ryujin::Mirrored< T >
class	ryujin::SelectView< dim, Number, MemorySpace, Object >

Typedefs
using	ryujin::selected_memory_space_t = std::conditional_t< have_separate_memory_spaces, dealii::MemorySpace::Default, dealii::MemorySpace::Host >
template<typename MemorySpace >
using	ryujin::other_space_t = std::conditional_t< std::is_same_v< MemorySpace, dealii::MemorySpace::Host >, dealii::MemorySpace::Default, dealii::MemorySpace::Host >

Enumerations
enum class	ryujin::TransferPolicy { ryujin::TransferPolicy::explicit_transfers , ryujin::TransferPolicy::implicit_transfers , ryujin::TransferPolicy::implicit_transfers_host_resident , ryujin::TransferPolicy::implicit_transfers_default_resident }

Functions
constexpr bool	ryujin::performs_implicit_transfers (const TransferPolicy policy)
template<int dim, typename Number , typename MemorySpace , typename Object >
auto	ryujin::make_select_view (const Object &object)

Variables
constexpr bool	ryujin::have_separate_memory_spaces
constexpr unsigned int	ryujin::warp_size

Detailed Description

This module contains classes and functions used for managing data mirrored between the host and default (device) memory spaces: the MirroredStorage base class implementing a unified interface for querying residency and transferring data between memory spaces, the TransferPolicy describing whether such transfers happen explicitly or implicitly, and the Mirrored convenience wrapper that maintains a single "POD style" object on both memory spaces.

Typedef Documentation

selected_memory_space_t

using ryujin::selected_memory_space_t = typedef std::conditional_t<have_separate_memory_spaces, dealii::MemorySpace::Default, dealii::MemorySpace::Host>

The selected memory space for the compute kernels.

Note

We switch between the compute memory spaces in this manner to have a clear toggle for selecting our custom SIMD-vectorized loops on the CPU (instead of relying on the fact that the default space will point to the host space if ryujin is configured without device support.

Definition at line 62 of file gpu.h.

other_space_t

template<typename MemorySpace >

using ryujin::other_space_t = typedef std::conditional_t<std::is_same_v<MemorySpace, dealii::MemorySpace::Host>, dealii::MemorySpace::Default, dealii::MemorySpace::Host>

A template alias that maps a given memory space to the respective other one: dealii::MemorySpace::Host maps to dealii::MemorySpace::Default and vice versa.

Definition at line 76 of file gpu.h.

Enumeration Type Documentation

TransferPolicy

enum class ryujin::TransferPolicy

strong

A policy describing how host/device memory transfers are performed when accessing a MirroredStorage object via view().

Enumerator
explicit_transfers	view<MemorySpace>() asserts that the selected memory space is resident; all transfers must be requested manually via copy_to_memory_space() and move_to_memory_space().
implicit_transfers	Requesting a read-only view triggers an implicit copy_to_memory_space() if the selected memory space is not resident (both memory spaces remain resident afterwards). Requesting a writable view triggers an implicit move_to_memory_space(): the data is copied over if necessary and the other memory space is deallocated and marked non-resident. Writable access invalidates the stale mirror.
implicit_transfers_host_resident	The same policy as implicit_transfers, but with the host memory space "pinned": operations that would deallocate the host storage, i.e., move_to_memory_space<dealii::MemorySpace::Default>() and requesting a writable view on the default memory space, are disallowed. The data thus always remains resident on the host memory space and pointers (and views) into the host storage remain valid. If the host and default memory spaces coincide the restriction is lifted: there is only a single allocation that no transfer can deallocate.
implicit_transfers_default_resident	The converse of implicit_transfers_host_resident: the default (device) memory space is pinned and move_to_memory_space<dealii::MemorySpace::Host>() as well as requesting a writable view on the host memory space are disallowed. If the host and default memory spaces coincide the restriction is lifted.

Definition at line 88 of file gpu.h.

Function Documentation

performs_implicit_transfers()

constexpr bool ryujin::performs_implicit_transfers ( const TransferPolicy  policy )

inlineconstexpr

Return true if the given transfer policy performs implicit memory transfers when a view is requested, i.e., for all implicit_transfers* policies.

Definition at line 139 of file gpu.h.

References ryujin::implicit_transfers, ryujin::implicit_transfers_default_resident, and ryujin::implicit_transfers_host_resident.

make_select_view()

template<int dim, typename Number , typename MemorySpace , typename Object >

auto ryujin::make_select_view

(

const Object &

object

)

Create a SelectView object, see the documentation of SelectView.

Definition at line 700 of file gpu.h.

Variable Documentation

have_separate_memory_spaces

constexpr bool ryujin::have_separate_memory_spaces

inlineconstexpr

Initial value:

=
      !std::is_same_v<dealii::MemorySpace::Host::kokkos_space,
                      dealii::MemorySpace::Default::kokkos_space>

A constexpr boolean that is true if the host and default (device) memory spaces are distinct. If ryujin is configured without device support then dealii::MemorySpace::Default coincides with dealii::MemorySpace::Host and no memory transfers are necessary.

Definition at line 29 of file gpu.h.

Referenced by ryujin::Postprocessor< Description, dim, Number >::compute(), ryujin::MeshAdaptor< Description, dim, Number >::compute_smoothness_indicators(), ryujin::Vectors::debug_poison_invalid_values(), ryujin::HyperbolicModule< Description, dim, Number >::prepare(), ryujin::SolutionTransfer< Description, dim, Number >::prepare_projection(), ryujin::SolutionTransfer< Description, dim, Number >::project(), ryujin::VTUOutput< Description, dim, Number >::schedule_output(), and ryujin::HyperbolicModule< Description, dim, Number >::step().

warp_size

constexpr unsigned int ryujin::warp_size

inlineconstexpr

Initial value:

=
      have_separate_memory_spaces ? 32
                                  : dealii::VectorizedArray<NUMBER>::size()

The "warp size": the number of consecutive row indices of the internal index range [0, n_internal_dofs) that a SparsityPattern bins together. All rows of a warp have the same stencil size and their matrix entries are stored contiguously (array-of-struct-of-array), see the documentation of the SparsityPattern class.

Note

Setting this to the wrong number severly degrades performance on GPU or CPU as the memory access is no longer consecutive.

Definition at line 46 of file gpu.h.

Referenced by ryujin::DoFRenumbering::export_indices_first(), ryujin::DoFRenumbering::inconsistent_strides_last(), and ryujin::DoFRenumbering::internal_range().