The IRMA Community
Newsletters
Research IRM
Click a keyword to search titles using our InfoSci-OnDemand powered search:
|
High Performance Storage for Big Data Analytics and Visualization
Abstract
Scientific Big Data being gathered at exascale needs to be stored, retrieved and manipulated. The storage stack for scientific Big Data includes a file system at the system level for physical organization of the data, and a file format and input/output (I/O) system at the application level for logical organization of the data; both of them of high-performance variety for exascale. The high-performance file system is designed with concurrent access, high-speed transmission and fault tolerance characteristics. High-performance file formats and I/O are designed to allow parallel and distributed applications with easy and fast access to Big Data. These specialized file formats make it easier to store and access Big Data for scientific visualization and predictive analytics. This chapter provides a brief review of the characteristics of high-performance file systems such as Lustre and GPFS, and high-performance file formats such as HDF5, NetCDF, MPI-IO, and HDFS.
Related Content
|
Renjith V. Ravi, Mangesh M. Ghonge, P. Febina Beevi, Rafael Kunst.
© 2022.
24 pages.
|
|
Manimaran A., Chandramohan Dhasarathan, Arulkumar N., Naveen Kumar N..
© 2022.
20 pages.
|
|
Ram Singh, Rohit Bansal, Sachin Chauhan.
© 2022.
19 pages.
|
|
Subhodeep Mukherjee, Manish Mohan Baral, Venkataiah Chittipaka.
© 2022.
17 pages.
|
|
Vladimir Nikolaevich Kustov, Ekaterina Sergeevna Selanteva.
© 2022.
23 pages.
|
|
Krati Reja, Gaurav Choudhary, Shishir Kumar Shandilya, Durgesh M. Sharma, Ashish K. Sharma.
© 2022.
18 pages.
|
|
Nwosu Anthony Ugochukwu, S. B. Goyal.
© 2022.
23 pages.
|
|
|