Reprints from my posting to SAN-Tech Mailing List and ...

2011/06/28

[san-tech][03218] LLNL Hyperion: Lustre と GPFS 比較レポート (IOR, mdtest)

Date: Tue, 28 Jun 2011 12:58:34 +0900
-----------------------------------------------
2010年 11月 15日に開催された

5th Petascale Data Storage Workshop, Supercomputing '10
  http://www.pdsi-scidac.org/events/PDSW10/index.html

のポスターセッション
POSTER SESSION PARTICIPANTS
  http://www.pdsi-scidac.org/events/PDSW10/postersession.html
に投稿された (Poster Submissions)

"Comparison of leading parallel NAS file systems on commodity hardware"
 Richard Hedges, Keith Fitzgerald, Mark Gary, D. Marc Stearman,
 Lawrence Livermore National Laboratory
  http://www.pdsi-scidac.org/events/PDSW10/resources/posters/parallelNASFSs.pdf
のテクニカルレポートが公開されました:

"Comparison of leading parallel NAS file systems on commodity hardware"
 Richard Hedges, Keith Fitzgerald, Mark Gary, D. Marc Stearman,
 Lawrence Livermore National Laboratory
 LLNL-TR-461793, Publication Date: 2010 Nov 08, System Entry Date: 2011 Jun 23
  http://www.osti.gov/bridge/product.biblio.jsp?query_id=0&page=0&osti_id=1016306&Row=0

Abstract
  "..... In this activity, we present the results of our tests of two
   leading file systems (GPFS and Lustre) on the same physical hardware.
   This hardware is the standard commodity storage solution in use at
   LLNL and, while much smaller in size, is intended to enable us to
   learn about differences between the two systems in terms of
   performance, ease of use and resilience. This work represents the
   first hardware consistent study of the two leading file systems that
   the authors are aware of."


Hyperion
  https://hyperionproject.llnl.gov/index.php
Hyperion Scalable Units
  https://hyperionproject.llnl.gov/hyperion_scalable_units.php
  https://hyperionproject.llnl.gov/images/hyperion_phase1.png
Hyperion Petascale IO Testbed
  https://hyperionproject.llnl.gov/hyperion_petascale_testbed.php
  https://hyperionproject.llnl.gov/images/hyperion_testbed.png

  576 node: Intel Harpertown LS420@2.5GHzx2(8 core), 8GB@21.6GB/s, 4x DDR
  576 node: Intel Nehalem E5530@2.4GHzx2(8 core), 12GB@64GB/s, 4x DDR
    Full Fat Tree InfiniBand 4x DDR
  GateWay nodes: InfiniBand 4x DDR + 10GbE x 1(2)

  10GbE Edge: Cisco Nexus 5020 (GateWay)
  10GbE Core: Cisco Nexus 7018
  10GbE Edge:  Arista 7148S (Storage)

  MDSs (NSD0): Nehalem E5530@2.4GHzx2(8 core), 10GbE, 4x SDR,
                      15K SASx16
  OSTs (NSDx)x4: Nehalem E5530@2.4GHzx2(8 core), 10GbE, 4x SDR

  DDN S2A 9550 4x SDR

この環境で
IOR (THROUGHPUT TESTING)
  http://sourceforge.net/projects/ior-sio
mdtest (METADATA PERFORMANCE TESTING, PER PROCESS METADATA SCALING STUDY)
  http://sourceforge.net/projects/mdtest/
を使用して比較しています。

様々な条件で比較しているので、興味のある方はレポートに目を通して下さい。
※128ノード程度でサチるようなので、報告最大ノード数は 128です。
※この検証は DDN 9550 1台だけですが、実際には沢山のストレージがあります。


Hyperionの構成については、例えば:

"PetaScale I/O Challenges: Hyperion and Sequoia"
 Matt Leininger, LLNL, OpenFabrics Workshop March 23, 2009
  http://www.openfabrics.org/archives/spring2009sonoma/monday/petascio.pdf
"Lustre on Hyperion"
 Marc Stearman, LLNL, Lustre User Group 2009
  http://wiki.lustre.org/images/6/60/LUGHyperion2009.pdf


Hyperionプロジェクトの趣旨については、以下を参照して下さい:

"The Hyperion Project: Partnership for an Advanced Technology Cluster Testbed"
 Mark Seager ; Matt Leininger. Lawrence Livermore National Laboratory
 LLNL-TR-403271, Publication Date: 2008 Apr 28,
  http://www.osti.gov/bridge/product.biblio.jsp?query_id=0&page=0&osti_id=938505
[san-tech][01384] Cisco Nexus:大規模 HPCテスト環境に納入、(2008/12/04)
[san-tech][01403] Re:  Cisco Nexus:大規模 HPCテスト環境に納入、(2008/12/24)

現在の Hyperionは、以下で少し紹介されているように Fusion-ioを導入して
います:

"Addressing the Challenges of Petascale Systems Deployment"
 Mark Seager, Lawrence Livermore National Laboratory
 Salishan Conference High-Speed Computing 2010, April 26 -29, 2010
  http://www.lanl.gov/orgs/hpc/salishan/salishan2010/pdfs/Mark%20Seager.pdf
[san-tech][02384] DOE LLNL Hyperion (Appro + Fusion-IO) & Virident PCIe SSDボードとか

> "Appro Deploys a World Class Linux Cluster Testbed Solution to LLNL
>  in Support of the Hyperion Project", 6/26/2010
>   http://www.appro.com/press/view.asp?Num=193
>
> "Lawrence Livermore Teams with Fusion-io to Re-define Performance Density"
>  June 14, 2010
>   http://www.fusionio.com/press/Lawrence-Livermore-Teams-with-Fusion-io-to-Re-define-Performance-Density/
>   http://www.fusionio.com/load/media-docsPress/rr6fj/Pressrelease_LLNL_Fusion_3.pdf
>
> "Nuke lab tests flashy HPC server cluster", 16th June 2010
>   http://www.theregister.co.uk/2010/06/16/appro_fusion_io_llnl/

Case Study, Fusion-io
"Lawrence Livermore National Laboratory Redefines High Performance
 Computing with Fusion Powered I/O"
 Last updated Jan-17-2011
  http://www.fusionio.com/case-studies/llnl/
  http://www.fusionio.com/load/-media-/1a7jm5/docsCaseStudies/FIO_LLNL_Book_v10_web.pdf

Fusion-io 月別アーカイブ: 5月 2011
  http://fusionio.wordpress.com/2011/05/
"Fusion-IOの国内事例情報まとめ", 5月 29, 2011
によると、Dellのサイトに上記ケーススタディの日本語訳があるようです。

0 件のコメント:

コメントを投稿