mercredi 9 décembre 2009

Erlang/OTP must be in your toolbox

I would like to convince you that there's no one-size-fits-all language. Building a concurrent application for thousands users should always be designed with grains of salt. Developers are now required to know more than one language because one doesn't fit all your requirements. Many languages emerge (Go, Clojure etc..). Some of them are mature but they are not yet mainstream language like Java (Haskell, Lisp ). They are not newsworthy but they deserve being better known. This diversity of language is a chance. I'd a like to convince you that Erlang is a great language. Let's get down to the nifty-gritty reasons why it is worth considering it. Imperative vs declarative When you build an application, you have 2 options: imperative language and/or functional programming language. I think the right option is when you write the What instead of the How. Using an imperative language enforces you to write expressions that more or less accurately indicate how you manage memory. In an object oriented language, encapsulating the behaviour and state in an object doesn't get rid off race conditions unless it is immutable or its state is shared with synchronization. A common strategy in imperative language is synchronization of threads. However, this often leads to deadlocks because two threads are locking resources that depend on each other. Instead of the What, you spend most of your time writing the How. In a declarative language, you can express complex logic with powerful declarative syntax. Pattern matching is a syntactic sugar that provides an excellent alternative to imperative test/cast paradigm. Passing an anonymous function as a parameter to another function is also a great way to combine logic. Function is the first-class component of a functional programming language. For example, parsing an XML tree with combinators using the high-order function construct makes your code much more concise and elegant. But there is more than constructs that makes your life easier. There is immutability. In functional programming, processes are non-blocking because they don't share memories. Erlang process don't share memories unlike threads. It has a concurrent structure. Building concurrent applications in Erlang is fitted like glove because you have concurrency without synchronization. Erlang vs other declarative languages Erlang without OPT is like Java without J2EE or Ruby without Rails. It would be a great language. However it would lack a robust container. Without OTP and its callback structure, you might have to spawn, register, send or recieve. OTP comes to the rescue. Just focus on the callback that is your business feature in a pure sequential code. OTP framework might, for example, take care of making the application fault tolerant or scalable, whereas the behavorial callback concentrates on the specific aspects of the problem. For you, OTP handles concurrency out of the box. For a month, i've been searching the killer language that I could use to build a scalable, fault-tolerant and fast chat system. At first, I prefered Scala and the Lift web framework. It sounds to me a tough option as Scala relies on a mature JVM and provides a nice concurrency model: actors. When David Pollak, the Lift founder decided to refactor the Scala actor and to develop from its own Lift actor to make Lift more stable I suddenly became skeptical. Is the Scala lightweight actor less stable than Erlang process? Scala is a nice hybrid language relying on long-live process : the Java threads. However, if I wanted to build a reliable chat system, the question is not whether I should use system process or Erlang runtime process but how many of them can I run on my host? How to solve the C10K problem? As the Erlang process are emulated in the Erlang runtime, you get a loose coupling with the OS capacity. The Yaws HTTP server in Erlang performs pretty well against the existing servers (Apache and Nginx). One can have up to 35 000 process as the other one is limited to 5000 sockets in a 32 bit UNIX machine. Furthermore Erlang programs can have linear speed up when they run on multicore computer. There are many real large scale system in Erlang such as RabbitMQ, Ejabberd just to mention a few. Amazon implemented its storage system "SimpleDB" in Erlang; a cluster of Ejabberd nodes daily serves millions of messages for the Facebook chat users. OTP is used in the telecom infrastructure of Ericsson ( see the case study of Motorola too. Some pretty fast-moving startups in the financial world have latched onto Erlang; for example, the Swedish www.kreditor.se. Conclusion Being a declarative language empowered by a battle-tested platform, Erlang is an excellent option to build distributed system and concurrent applications. I didn't digg into the features of OTP (reliability, fault-tolerance, hot code swapping etc..) because an item would not be enough. I didn't cover the drawbacks: crappy record syntax, string being a list. Anyway, the Web requirements for more interactive and scalable system make Erlang an outstanding language. Nowadays a lot of web framework in Erlang are emerging (Nitrogen, Chicago, ErlyWeb, MochiWeb). I'm looking forward to seeing how Erlang will be adopted in the coming years.