#	Pagina
attuale pagina	/open-h2020/projects/197944/results.html

Report

Teaser, summary, work performed and final results

Periodic Reporting for period 3 - ExaHyPE (An Exascale Hyperbolic PDE Engine)

Teaser

Summary

ExaHyPE developed an engine to solve hyperbolic systems of partial differential equations (PDE) on exascale supercomputers. ExaHyPE particularly addressed problems in Seismology (e.g., earthquake simulation) and Astrophysics (systems of black holes or neutron stars, gravitational waves). As an â€œengineâ€ ExahyPE supports a wide range of problems: solvers for more than 20 hyperbolic PDE systems have already been realised. Thus, ExaHyPE enables groups of computational domain scientists to realize grand challenge simulations in much quicker â€time to scienceâ€.
To address the challenges of exascale, ExaHyPE developed a novel, communication-avoiding high-order discontinuous-Galerkin scheme with high-order time stepping. At shocks and critical regions, the solver falls back to a robust Finite-Volume limiter. We feature tree-structured dynamical adaptive mesh refinement and express all numerical steps via task-based paradigms. We developed agile load balancing via lightweight work stealing across compute nodes and explored replication-based resiliency and the impact of a-posteriori limiting on detection and mitigation of soft faults.
As one of two demonstrators, we provide ExaSeis, for seismology, which includes (1) a linear model on curvilinear meshes, where the mesh is adapted to topography, material layers or fault planes, and (2) a novel non-linear diffused-interface model that expresses arbitrarily complex domains on Cartesian grids via a color function. As a second demonstrator, we provide solvers for the Einstein field equations of general relativity in first-order CCZ4 formulation, coupled to the general relativistic MHD equations. We thus enable relativistic simulation of black holes and neutron stars, which are being further developed towards mergers of binary systems.

Work performed

In the first project year, we developed a prototype of the ExaHyPE engine that combined the high-order numerical scheme with the tree-structured Cartesian meshing paradigm. A flexible programming interface was established that offered hybrid shared & distributed parallelism transparent to the user. A code generator provides optimised compute kernels tailored to the specific PDE system and discretisation order.
In 2016 and 2017, the engine prototype was extended to a fully functional engine for solving hyperbolic PDE systems. An Application Specific Programming Interface (ASPI) was developed that allows users to easily express specific systems of PDEs via â€œuser functionsâ€. The engine implements linear and non-linear ADER-DG schemes on dynamically adaptive Cartesian-structured meshes. All schemes are parallelised using MPI and Intel Threading Building Blocks as programming model.
For the astrophysics demonstrator, we developed a novel, provably strongly hyperbolic first-order reduction of the conformal and covariant Z4 formulation of the Einstein field equations (FO-CCZ4). For seismology, we defined and prototypically implemented three grand-challenge use-cases based on curvilinear and diffused interface models, both avoiding major bottleneck in many applications of computational seismology: the need to resolve topography and sharp material interfaces via feature-preserving meshes.
Results of 2018 and 2019 include the further development of ExaHyPE\'s Toolkit and Kernel Generator into a role-oriented code generator with separation of concerns for application, algorithms and optimisation experts. We realised lightweight task-offloading for reactive load balancing across compute nodes. This is complemented by team-based replication of MPI ranks (based on a tailored infrastructure, teaMPI) to improve resilience of capability simulations.
We developed a new numerical approach to discretise curl-type involutions to treat instabilities arising in the coupled FO-CCZ4+GRMHD models. The added curl-cleaning terms increase the size of the PDE system to more than 100 quantities. Work on the Godunov-Peshkow-Romenski model allows to model linear and nonlinear elasto-plastic solids with crack formation, as well as viscous and ideal fluids at the same time within one unified governing PDE system. Alongside the diffused interface model for seismology, we developed a diffuse interface method for compressible flows over fixed and moving solids, which substantially widens the range of ExaHyPE applications towards gas dynamics in complex geometries and fluid-structure-interaction problems.

Final results

We have realised the first engine to solve a wide range of hyperbolic PDE systems using arbitrary high-order discretisation in space and time on dynamically adaptive meshes. Several of the developed numerical models are novel contributions, esp. the first-order formulations of the Einstein equations (FO-CCZ4, provably hyperbolic) and the diffuse/sharp interface methods for seismic wave propagation. The ADER-DG schemes for the CCZ4 formulation, for the Godunov, Peshkov and Romenski model, for the diffuse/sharp interface method and several physics-based improvements of the ADER-DG scheme on curvilinear meshes (and PML boundary conditions) are novel contributions of the project. The coupling of FO-CCZ4 and GRMHD equations for simulating neutron stars revealed instabilities that were cured by development of a novel curl-cleaning approach.
The recent detection of gravitational waves from the merger of binary black holes and neutron stars (the latter with an electromagnetic counterpart) motivates highly accurate simulations. Simulation tools that cover the full range of physics, i.e., fully relativistic modeling, magnetic fields and viscous hydrodynamics, are not yet available. Existing state-of-the-art simulation software only offers a subset of models and only provides limited adaptive mesh refinement.
The new seismic wave propagation models allow to simulate scenarios with highly complex geometries such as numerical scenarios for the AlpArray, Europeâ€™s currently largest seismology experiment. The ExaSeis code, developed on the basis of ExaHyPE will be further developed as part of the ChEESE Centre of Excellence funded via Horizon 2020.