Date: Fri, 16 Dec 2011 21:02:37 +0900
--------------------------------------------------
[san-tech][03429] Re: PLFS, a Parallel Log-structured File System, Version2..1, 2011-10-13
Date: Thu, 22 Dec 2011 09:32:30 +0900
> "Los Alamos and EMC collaborate to advance high-performance computing"
> December 21, 2011
> http://www.lanl.gov/news/releases/los_alamos_emc_collaborate_to_advance_high_performance_computing.html
--------------------------------------------------
ちょっとマニアックかもしれませんが、PLFS, a Parallel Log-structured File System
の開発が進行しています:
PLFS: A Checkpoint Filesystem for Parallel Applications
http://institute.lanl.gov/plfs/
"PLFS: A Checkpoint Filesystem for Parallel Applications"
John Bent, Garth Gibson, Gary Grider, Ben McClelland,
Paul Nowoczynski, James Nunez, Milo Polte, Meghan Wingate
SC '09
http://institute.lanl.gov/plfs/plfs.pdf
Abstract
....
"... we have developed a parallel log-structured file system, PLFS,
which is positioned between the applications and the underlying
parallel file system. PLFS remaps an application’s write access
pattern to be optimized for the underlying file system. Through
testing on Panasas ActiveScale Storage System and IBM's General
Parallel File System at Los Alamos National Lab and on Lustre at
Pittsburgh Supercomputer Center, we have seen that this layer of
indirection and reorganization can reduce checkpoint time by up to
several orders of magnitude for several important benchmarks and
real applications."
PLFS Source Code
http://sourceforge.net/projects/plfs/
http://sourceforge.net/projects/plfs/files/
plfs-2.1.tar.gz: 2011-10-13
README.installから 2行:
1) To install PLFS as a FUSE filesystem.
2) To install PLFS as a ROMIO ADIO layer for MPI-IO.
README.posix冒頭:
"The primary goal of PLFS is to support very high bandwidth parallel
IO for HPC large scale parallel applications especially for N-1
write workloads."
"A secondary goal is to be as POSIX compliant as possible without
sacrificing the primary goal. There are several known cases where
PLFS is intentionally not POSIX compliant in order to satisfy the
primary goal:"
(snip)
==================================================
2011/12/17
PLFSは N-1書き込みバンド幅を増加するために、並列ファイルシステムの上で動く
ミドルウェア、元データを PLFSで Log Structuredに再配置して、並列ファイルシステム
への書き出しバンド幅をかせぐ。書き込みバンド幅に対して読み出しは弱いと思われる
ので、例えばチェックポイントファイルとか
"PLFS: A Fast Checkpoint Filesystem"
John Bent, Garth Gibson, Gary Grider, Ben McClelland, Paul Nowoczynski,
James Nunez, Milo Polte, Meghan Wingate
http://www.pdsi-scidac.org/events/PDSW10/resources/posters/plfs-r10.pdf
※Lustreでの比較グラフあり
"PLFS and HDFS: Enabling Parallel Filesystem Semantics In The Cloud"
Milo Polte, Esteban Molina-Estolan, John Bent, Garth Gibson,
Carlos Maltzahn, Maya B. Gokhale, Scott Brandt
http://www.pdsi-scidac.org/events/PDSW10/resources/posters/PLFS-HDFS-svd10.pdf
POSTER SESSION PARTICIPANTS
http://www.pdsi-scidac.org/events/PDSW10/postersession.html
5th Petascale Data Storage Workshop (PDSW)
Supercomputing '10, November 15, 2010
http://www.pdsi-scidac.org/events/PDSW10/index.html
0 件のコメント:
コメントを投稿