Insights from 150 worldwide experts to achieve exascale
Starting from the first European initiative for exascale software, EESi1, Europe has developed a unique network of more than 150 academic and industrial experts, including both scientific software developers and users, to assess the impact of Exascale computing on society, environment and technology, and lead the way.
With EESI1, the first worldwide cartography of developments was issued. Furthermore, a global Exascale software stack roadmap and an Exascale application roadmap, including fundamental research and industrial grand challenge applications was proposed.
Insights on Europe strengths and weaknesses have enabled to set priority actions, define the type of cooperation structure that should be implemented between Europe and the international community.
In 2013 and 2014, EESI2 has published its updated vision of what need to be accomplished to build an exascale capacity for Europe.
Exascale is a disruptive computing technology that addresses huge economic, societal and scientific challenges. Investment in this capability will give massive returns for Europe.
The roadmap foundation for Exascale in Europe has been set in 2011 by the European Exascale Software Initiative (EESI1).
An important outcome of experts that have contributed to this initiative was the recommendation to address the overall challenge at a worldwide level.
A close collaboration of EESI has been established with experts from Europe, and international stakeholders and luminaries from the United States and Japan (eg International exascale Software Project of The Department of Energy, and Satoshi Matsuoka executive member of IESP21).
Exchange of thoughts during international workshops and conferences has helped identify where Europe stands in the worldwide exascale race:
- A set of HPC computing systems. This includes the PRACE multi-petaflopic research infrastructure serving more than 22 countries and large industry corporation systems such as Total, EDF or Airbus.
- community organizations supporting various leading scientific fields
- Applications, software and applied mathematics: more than 80% of parallel software applications and codes used in Europe is created by solid European teams
- lack of structuration for some scientific domains eg though numerical analysis skills are strong, there is no European package or team.
- Lack o f participation to new programming standards such as MPI, OpenMP
- Little scientific and technical critical mass on some strategic software components like Operating System or compilers. Moreover, Europe lacks proprietary database in some domains thus has difficult access to data without a unified layer
- Ageing applications that need redesign in some leading domains such as Materials/Molecular Mechanics, Climate models, quantum chemistry
Some technical issues for Exascale computing, including Extreme Computing and Big Data, have been identified by EESI1.
At the level of the simulation environment, issues include a unified framework, multi-physics simulation, mesh generation tools and standardized parallel I/O and data management
At the level of codes/applications, five domains that need attention include: the efficiency of numerical methods, algorithms or solvers, the coupling between stochastic and deterministic methods, the meshless methods and particle simulation, the program scalability and fault tolerance, and the standard programming models for multi-level parallelism.
As conclusion, the development of Exacale eco-systems in Europe was seen as an urgent need to sustain leadership in such domains as research, notably in the new promising fields of computational materials design, device design, engineering applications (CFD, seismic, …), molecular dynamics, bioinformatics or life science where Europe has an outstanding position in scalable codes and expertise.
Building an exascale capability requires a sustainable, long term and coordinated effort. EESI1 has recommended to:
- Fund strategic projects where Europe is strong and able to federate significant critical mass.
- Insure the coordination of European efforts with the rest of the world. In particular, on strategic areas where Europe lack of critical mass and in worldwide initiatives in those that Europe has more value
- Reinforce multi-disciplinary HPC Centres providing support in terms of code development, porting and optimization as well as algorithm development
- Foster the development of community organizations (CECAM, ENES, EBI…), forming
networks between scientific groups, disseminating knowledge within specific scientific domains and identifying demanding and grand challenge problems in application domains.
Link with community specific co-design centres should be ensured.
- Promote open source development while enabling commercial exploitation.
- Support commercial or privately owned software improvement.
This roadmap includes to develop strong and focused initiatives towards exascale computing, including specific training and education programs.
Additionally, Europe should encourage the development of Open Source solutions to foster international collaborations and the emergence of international de facto standards, enabling commercial exploitation.
With such a roadmap, the European commission will enable the deployment and use of Exaflop computing power in Europe by the end of this decade
Discover dissemination activities of the EESI experts notably Prace report.
PRACE Scientific Case Update for High Performance Computing in Europe (2012 – 2020)
Since the first vision of the European Exascale Software Initiative in 2011, experts have developed a periodic update of vision, roadmap and recommendations for an effective implementation of Exascale in Europe. The EESI2 vision clearly recognizes the imperative need for breakthrough on hierarchical algorithms and more technologies.
The EESI2 Initiative is clearly oriented toward the development and implementation of efficient Exascale applications, algorithms and software for enabling the emergence of a new generation of data intensive and extreme computing applications.
In 2013, EESI2 had published the first vision update. Eight key R&D programs for an exascale roadmap have been identified:
- 5 absolutely critical programs: ultra scalable algorithms, resilience, Big Data, couplers, high productivity programming models.
- 3 high priority programs: mini apps, software engineering methods for high- performance computing, and Verification, Validation and Uncertainty Quantification
EESI has refined its vision, inspired by the various explorations, observations of worldwide recent installations in Europe, Asia and in USA of 10 (and more) Petaflops computers, and by the feedback of several applications and tests running on full configurations of these systems.
Outcomes are coherent with those of international R&D funded programs on Exascale.
The following points are on the critical path to Exascale Computing:
- The use of hierarchical algorithms that reduce communications and tasks synchronizations
- The use of multi-physics methods that do not need or minimize data transfers and include multi scaling and parallel space-time methods
- The reshaping of operating systems and management tools such as MPI and OpenMP and mesh generation tools to the new developed algorithms
- The use of in situ data processing
In 2014, given the ultra complexity and inter-dependency of extreme computing and extreme data, EESI has changed over to a new approach, data centric, to pave the way toward exascale computing
New thinking is required to develop new programming models, new algorithms, new tools, new data processing methods … not only bigger than the present ones, which will remain useless, but far beyond the required innovation. Exascale creates fundamental new opportunities, but it also brings fundamental new challenges. Exascale means exaflops and exabytes. The data centric approach is a must.
The new roadmap and the consecutive recommendations are gathered into three large pillars:
Europe clearly has strengths (applications, scalable algorithms, couplers…) but at the same time Europe is late on some Exascale key issues (languages, programming tools …). Thus the principles underlying EESI2 vision and recommendations are:
- Exascale is not only a “bigger HPC”. There is an urgent need for new specific and disruptive ultra- scalable improvements in order to realise its full potential
- Extreme computing and Extreme Data should be tackled simultaneously.
At Exascale, Extreme Computing and Extreme Data (or Big Data) are intrinsically linked since supercomputers become mandatory to analyze efficiently huge flows of data generated by large scale instruments or by massive complex simulations.
Exascale applications will be efficient only through developments by multidisciplinary teams, optimizing the interactions between architecture (nodes, cores, memories, interconnect, power, resilience …), algorithms (programming, ultra scalable numerical methods, asynchrony, fault tolerance …), and applications (discretization of problems, engineering tools, data processing …).
- It is urgent that Europe funds large projects focusing on complex specific Exascale challenges in particular in the domains where the EU has established strengths.
Read the detailed report