The New Face of Open Source: Facebook
Published 14:07, 03 February 10
Without doubt, one of the most extraordinary developments in recent years has been the rise of Facebook - not just as the most popular social network, but increasingly as a force to reckoned with in the world of computing, perhaps soon on the scale of Microsoft and Google. This makes its relationship to free software something of more than passing interest.
As it happens, Facebook is built on free software, as this post notes:
From the day Mark Zuckerberg started building Facebook in his Harvard dorm room in 2004, the site has been built on common open source software such as Linux, Apache, memcached, MySQL, and PHP.
With HipHop we've reduced the CPU usage on our Web servers on average by about fifty percent, depending on the page. Less CPU means fewer servers, which means less overhead. This project has had a tremendous impact on Facebook. We feel the Web at large can benefit from HipHop, so we are releasing it as open source this evening in hope that it brings a new focus toward scaling large complex websites with PHP. While HipHop has shown us incredible results, it's certainly not complete and you should be comfortable with beta software before trying it out.
Here's the background to HipHop's development, and what it does:
One night at a Hackathon a few years ago (see Prime Time Hack), I started my first piece of code transforming PHP into C++. The languages are fairly similar syntactically and C++ drastically outperforms PHP when it comes to both CPU and memory usage. Even PHP itself is written in C. We knew that it was impossible to successfully rewrite an entire codebase of this size by hand, but wondered what would happen if we built a system to do it programmatically.
Finding new ways to improve PHP performance isn't a new concept. At run time the Zend Engine turns your PHP source into opcodes which are then run through the Zend Virtual Machine. Open source projects such as APC and eAccelerator cache this output and are used by the majority of PHP powered websites. There's also Zend Server, a commercial product which makes PHP faster via opcode optimization and caching. Instead, we were thinking about transforming PHP source directly into C++ which can then be turned into native machine code. Even compiling PHP isn't a new idea, open source projects like Roadsend and phc compile PHP to C, Quercus compiles PHP to Java, and Phalanger compiles PHP to .Net.
Needless to say, it took longer than that single Hackathon. Eight months later, I had enough code to demonstrate it is indeed possible to run faster with compiled code. We quickly added Iain Proctor and Minghui Yang to the team to speed up the pace of the project. We spent the next ten months finishing up all the coding and the following six months testing on production servers. We are proud to say that at this point, we are serving over 90% of our Web traffic using HipHop, all only six months after deployment.
That's 90% of “over 400 billion PHP-based page views every month” according to the same post, or over 10 billion PHP-based pages views every day, on average.
What this means is that alongside mega-sites like Google, rumoured to have maybe millions of machines running a custom Linux kernel, we now have another amazing demonstration that open source not only scales, but that it scales way beyond anything else out there.
Whatever your views of social networks in general, we can be grateful that Facebook is turning into one of the key open source companies, and – just as importantly – a further demonstration that free software not only runs most of the Internet's infrastructure – the DNS, Web and email servers - but also many of the ecommerce giants that make use of it.