SAGE: Percipient Storage for Exascale Data Centric Computing

•A storage system, termed “SAGE”, capable for handling extreme volumes of data suitable for Exascale supercomputers.•SAGE is driven by an object storage software technology, Mero, which is fundamentally architected from scratch for Exascale unlike many existing storage software infrastructures.•SAGE...

Full description

Saved in:

Bibliographic Details
Published in:	Parallel computing 2019-04, Vol.83, p.22-33
Main Authors:	Narasimhamurthy, Sai, Danilov, Nikita, Wu, Sining, Umanesan, Ganesan, Markidis, Stefano, Rivas-Gomez, Sergio, Peng, Ivy Bo, Laure, Erwin, Pleiter, Dirk, Witt, Shaun de
Format:	Article
Language:	eng
Subjects:	Clovis Mero MPI I/O MPI streams Object storage PGAS I/O SAGE architecture
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Description
Summary:	•A storage system, termed “SAGE”, capable for handling extreme volumes of data suitable for Exascale supercomputers.•SAGE is driven by an object storage software technology, Mero, which is fundamentally architected from scratch for Exascale unlike many existing storage software infrastructures.•SAGE handles multiple tiers of storage device technology types including Non-Volatile Memories.•The SAGE system is capable of handling computational offloading on behalf of user applications.•The SAGE system works with PGAS and MPI programming frameworks, some results of which are presented in this work. We aim to implement a Big Data/Extreme Computing (BDEC) capable system infrastructure as we head towards the era of Exascale computing - termed SAGE (Percipient StorAGe for Exascale Data Centric Computing). The SAGE system will be capable of storing and processing immense volumes of data at the Exascale regime, and provide the capability for Exascale class applications to use such a storage infrastructure. SAGE addresses the increasing overlaps between Big Data Analysis and HPC in an era of next-generation data centric computing that has developed due to the proliferation of massive data sources, such as large, dispersed scientific instruments and sensors, whose data needs to be processed, analysed and integrated into simulations to derive scientific and innovative insights. Indeed, Exascale I/O, as a problem that has not been sufficiently dealt with for simulation codes, is appropriately addressed by the SAGE platform. The objective of this paper is to discuss the software architecture of the SAGE system and look at early results we have obtained employing some of its key methodologies, as the system continues to evolve.
ISSN:	0167-8191 1872-7336 1872-7336