Reprints from my posting to SAN-Tech Mailing List and ...


[san-tech][03236] Realtime Hadoop usage at Facebook, SIGMOD2011

Date: Fri, 01 Jul 2011 16:18:35 +0900
Facebook技術者による、Hadoopの Peta-byte scale規模での Realtime Workload対応
についての解説 Blogです (ACM SIGMOD 2011で発表):

"Realtime Hadoop usage at Facebook -- Part 1"
 May 17, 2011, HDFS, Dhruba Borthakur, Facebook
"Realtime Hadoop usage at Facebook -- Part 2 - Workload Types"
 May 28, 2011, HDFS, Dhruba Borthakur, Facebook

  "Facebook recently deployed Facebook Messages, its first ever
   user-facing application built on the Apache Hadoop platform. It uses
   HDFS and HBase as core technologies for this solution. Since then,
   there are many more applications that have started to used HBase.
   We have gained some experience in deploying and operating HDFS and
   HBase at peta-byte scale for realtime-workloads and decided to
   write a paper detailing some of these insights. This paper will be
   published in SIGMOD 2011." (Part 1)

  "You can find the full paper here later, but here are some highlights:"
※現時点 (2011/07/01) ではこちらのサイトでは公開されていません。

  "Before deciding on a particular software stack and whether or not to
   move away from our MySQL-based architecture, we looked at a few specific
   applications where existing solutions may be problematic. These use
   cases would have workloads that are challenging to scale because of
   very high write throughput, massive datasets, unpredictable growth,
   or other patterns that may be difficult or suboptimal in a sharded
   RDBMS environment." (Part 2)

SIGMOD 2011での発表ペーパー
"Apache hadoop goes realtime at Facebook"
 Dhruba Borthakur (facebook), et al.

Proceedings of the 2011 international conference on Management of data

2011 ACM SIGMOD/PODS Conference, June 12-16, 2011

Keynote 1: James Hamilton, Amazon Web Services
"Internet Scale Storage"
の講演資料は、Hamilton氏がご自身の Blogで公開しています。
"SIGMOD 2011 in Athens"
 2011/06/23, Perspectives, James Hamilton's Blog

ACM SIGMOD (Special Interest Group on Management of Data)


"Visualizing Facebook's PHP Codebase"
 2011/5/31, Greg Schechter, Software engineer at Facebook

  "Visualizations often are a great tool for understanding code
   interdependencies, especially when you're facing the challenge of
   determining interdependencies among more than 10,000 code modules.
   That happens to be about the number of modules that comprise the
   Facebook front-end code."
  "We released this visualization internally for the Facebook
   engineering team to consult and gain insight into module
   granularity, dependency management, and the like. As with all
   Facebook code, this is a work in progress, so we hope to add more
   value to it as it evolves."

0 件のコメント: