Tuesday, July 11, 2006

The Hitchhiker’s Guide to Erlang

If you've heard about Erlang and you want to use Erlang to build a website site but you're not sure where to start or where to find good online resources for Erlang web development, I know how you feel. I went through it myself. However, I stuck with it, did my share of Googling and mailing list questioning, and over time I got a good sense of how to navigate the Erlang world. I hope I can make your journey easier by giving you a few pointers.

First, let's get off to a quick start.

- download Erlang/OTP
- configure, make, make install.
- download Yaws, "The" Erlang web server.
- configure, make, make install
- run yaws from the command line
- create /tmp/foo.yaws and open it in a text editor
- type the following

my first Yaws page

out(A) ->
{ehtml, {p, [], "hello world!"}}.

- open your browser and go to http://localhost:8000/foo.yaws.

Here ya go! Now you're officially among the ranks of Erlang web developers. :)

Let's now take a few steps back. If you're reading this, you must have heard about Erlang and how its unique strengths let you build scalable, distributed, highly-availabile applications with much less effort than other languages. You want to use Erlang's power to build a scalable web application, but you've probably noticed that Erlang hasn't been as widely adopted by the web developer community as other solutions (e.g. Ruby on Rails), and that there aren't too many blogs and tutorials about Erlang web development online.

There's an Erlang reference book, Concurrent Programming in Erlang , which I admit I haven't read, but from the table of contents I see it doesn't discuss web development. You can find a couple of articles about Erlang, but they are mostly introductions Erlang as a language and none of them shows you how to use Erlang to quickly whip up a few web pages, which is probably what you want to with the least amount of pain.

None of this means you should follow the crowd and go to other languages just because they are somewhat more accessible and more developers are using them. I assure you that with some effort you'll get all the reference and help you need, and in a little while you'll be able to build the kind of backends that make Google seethe with envy.

Ok, maybe that was a slight exaggeration, but there's always a chance you're the next Larry Page, isn't there? :)

Alright, let's hit the road. Your first destination should be Open Source Erlang's official homepage. Every beginner should do the Getting Started Guide, and then the Language Reference.

As you're going through these manuals, make sure you have the Erlang shell open so you can write some short code samples. Type help() in the shell to see the list of commands.

Emacs is the "standard" development tool (the OTP distribution comes with Emacs language files), but if you've been spoiled by Eclipse you'll be happy to get your hands on Erlide.

When you think you have a good bearings on the basics, you may want to take a look at some code examples (I find that reading examples is often the best way to learn).

Now that you feel comfortable coding in Erlang and you're ready to experiment with some more web development, you should visit the Yaws webpage. This page has many useful instructions and examples on how to build dynamic web application with Yaws. If you understand Erlang, and you've done some web development, Yaws is pretty easy to learn. Yaws doesn't have a full MVC framework like Ruby on Rails, but it does have some nice features that give you similar abstractions: ehtml for views, appmods for controllers, Mnesia for the model. The Yaws distribution comes with some examples and you'll pick up some good methodologies by looking at their source code.

Build a few dynamic pages. You'll feel that tingling satisfaction when you see your Erlang code in action.

You should know that Yaws is a great web server for AJAX apps. Yaws has a JSON-RPC module, and even better, the next version of Yaws will support haXe remoting, hacked by yours truly. Read my previous blog posting about Yaws's unique strengths for Comet-based apps. IMO, this is the future of webapps, and Erlang + Yaws lead the competition by a wide margin in this area.

After you've made a few pages, it's a good time read the efficiency guide and programming conventions document, which admittedly aren't super exciting but it's good to know this stuff. Make sure you understand the cost of common operations. When you have a good bearings on the language, you should browse through the module list in the main documentation site to get a feel of what's available with the standard distribution

When you feel like you've done all you can independently and it's time to get some help from (gasp!) actual humans, you have two options: the erlang-questions mailing list and the erlyaws mailing list. erlang-questions is the place to go for general questions and erlyaws is more focussed on web development. Very knowledgeable people -- among them some of Erlang's creators -- will answer your questions on both mailing lists.

Read the Mnesia User's Manual. Mnesia is a distributed database written in pure Erlang and many Erlang apps use it. If your app doesn't require storing many gigabytes of data, Mnesia is a great solution. Keep in mind that Mnesia disc storage has a couple of issues. These may not be deal breakers, but if you have large datasets and you prefer to use a different disc storage engine, I recommend the MySQL or Postgres driver.

If you want to get a truly in-depth understanding of Erlang, you must read Joe Armstrong's PHD thesis. It will give you the overall picture on Erlang's history and the decisions that have shaped its design. It also covers basically every aspect of the language and explains how to to design fault tolerant systems in Erlang. I recommend this paper to every new Erlang developer. (Just don't panic when you read the section about Ericsson's banning Erlang from new projects. This stuff is ancient history.)

As you're building your app, you should know about Jungerl, an open source "Jungle of Erlang Code" -- a fitting description. Look at the list of modules because some of them may be useful to you.

If you want to explore even more exotic destinations, check out Joel Reymont's blog, in which he tells the story of building a scalable poker server in Erlang, and also Joel's DevMaster article about it. The source for his poker backend is available as well.

You can always learn a lot from looking at source code of experienced Erlang hackers. A few good sources I found are Yaws, Jungerl, Joel's OpenPoker server and ejabberd, a top notch Jabber server written in Erlang. ejabberd has makes use of MySQL/Postgres, and even has an abstraction layer for both drivers, which is useful if you wish to use these databases.

Well, I hope you find this useful. If there's anything you think I left out, please let me know. If you want to read more of my ramblings about Erlang, check out my previous blog postings about it or stay tuned for some future ones. Otherwise, stop wasting your time here and go build that Erlang-powered Google killer!


Frank said...

You list OpenPoker as a good source, even though it was written by someone without any previous experience, but you don't list Yaws? Foolishness.

yariv said...

Yes, I will add Yaws to the list. I didn't think about it for some reason. It's absent-mindedness, not foolishness :) I did think, though, that the OpenPoker code is interesting because it is accompanied by a blog and an article that describe its development.

Frankie said...

This is brilliant. It should be dugg. Or digged. Or digg dugg...err...yeah.

sander said...

Regarding ejabberd. There also exist patches that might be interesting:
* patches for LDAP (LDAPS, write support and more features)
* Yawspack to embed Yaws into ejabberd including useful software
* MSSQL support
* XML-RPC server
* active directory authentication

Jordan said...

Thank you so much for this! Keep at it!