Date: Fri, 01 Jul 2011 16:18:35 +0900
Facebook技術者による、Hadoopの Peta-byte scale規模での Realtime Workload対応
についての解説 Blogです (ACM SIGMOD 2011で発表)：
"Realtime Hadoop usage at Facebook -- Part 1"
May 17, 2011, HDFS, Dhruba Borthakur, Facebook
"Realtime Hadoop usage at Facebook -- Part 2 - Workload Types"
May 28, 2011, HDFS, Dhruba Borthakur, Facebook
"Facebook recently deployed Facebook Messages, its first ever
user-facing application built on the Apache Hadoop platform. It uses
HDFS and HBase as core technologies for this solution. Since then,
there are many more applications that have started to used HBase.
We have gained some experience in deploying and operating HDFS and
HBase at peta-byte scale for realtime-workloads and decided to
write a paper detailing some of these insights. This paper will be
published in SIGMOD 2011." (Part 1)
"You can find the full paper here later, but here are some highlights:"
※現時点 (2011/07/01) ではこちらのサイトでは公開されていません。
"Before deciding on a particular software stack and whether or not to
move away from our MySQL-based architecture, we looked at a few specific
applications where existing solutions may be problematic. These use
cases would have workloads that are challenging to scale because of
very high write throughput, massive datasets, unpredictable growth,
or other patterns that may be difficult or suboptimal in a sharded
RDBMS environment." (Part 2)
"Apache hadoop goes realtime at Facebook"
Dhruba Borthakur (facebook), et al.
Proceedings of the 2011 international conference on Management of data
2011 ACM SIGMOD/PODS Conference, June 12-16, 2011
Keynote 1: James Hamilton, Amazon Web Services
"Internet Scale Storage"
"SIGMOD 2011 in Athens"
2011/06/23, Perspectives, James Hamilton's Blog
ACM SIGMOD (Special Interest Group on Management of Data)
"Visualizing Facebook's PHP Codebase"
2011/5/31, Greg Schechter, Software engineer at Facebook
"Visualizations often are a great tool for understanding code
interdependencies, especially when you're facing the challenge of
determining interdependencies among more than 10,000 code modules.
That happens to be about the number of modules that comprise the
Facebook front-end code."
"We released this visualization internally for the Facebook
engineering team to consult and gain insight into module
granularity, dependency management, and the like. As with all
Facebook code, this is a work in progress, so we hope to add more
value to it as it evolves."