Amazon SimpleDB Runs on Erlang

I just came across this blog post: http://www.satine.org/archives/2007/12/13/amazon-simpledb/. The author got the scoop that Amazon's just-released SimpleDB runs on Erlang. This is actually the second Amazon Web Service that runs on Erlang -- at least, if the rumors I've heard that Amazon SQS was built with Erlang as well are true (update: I found the reference in this article).

It's pretty cool knowing that you can use the same language that powers some of the massive systems that Amazon and Ericsson have built to whip up a blog app just as easily as in Ruby or Python. (But if you built it in Erlang/ErlyWeb, you could also use Comet to make the blog update itself in real time, resting assured that horizontal scalability is just a matter of reconfiguring your Mnesia schema :) ).

Amazon SimpleDB is a game changer. It fills the last hole in making Amazon Web Services a complete environment for elastic, scalable web application hosting: the need for a fast, reliable data store. Together with S3 and EC2, SimpleDB makes it feasible for small startups to scale like the big companies but without the operational overhead. What a great set of products from Amazon.

ErlyWeb vs. Ruby on Rails EC2 Benchmarking Strangeness

I've been running some more benchmarks for the ErlyWeb vs. Ruby on Rails EC2 Performance Showdown. The results I've gotten are very strange -- so strange, in fact, that I'm not going to "officially" publish anything before I run all the tests one more time.

What's strange about the results? I'll give you a quick glimpse, but please keep in mind that none of these observations are the official benchmarks results and they may all be invalidated in the next run.

- Recompiling the Erlang files *without* HiPE seemed to *increase* max performance (in requests/sec) by ~10%.
- Running Yaws with kernel poll enabled ("+K true" passed to erl) *decreased* the max ErlyWeb performance to 410 (~41% decrease).
- No matter how many Mongrels I started (I tried 1,3,5, and 10) behind Pen (running on the same server), the max performance of Rails was ~18% lower than when it was running a on single Mongrel without Pen (111 requests/sec).

This across-the-board strangeness calls for another run. I'll try to have the results sometime in the next few days. In the meantime, if anybody else wants to take a stab at the these benchmarks, it would be interesting to see how our results compare.

By the way, although the effects of the changes I made were different from my expectations, when I ran the same tests the results were similar to the results I got last time.

ErlyWeb vs. Ruby on Rails EC2 Performance Showdown

A few people have asked me to run benchmarks comparing ErlyWeb and other frameworks, especially Ruby on Rails. Benchmarking is pretty boring, which is why I haven't benchmarked ErlyWeb until now. Plus, although I was pretty confident that ErlyWeb outperforms Rails, I didn't think proving it mattered that much. People who love Rails generally don't care much about raw performance, and people who love Erlang and functional programming don't need much convincing to use ErlyWeb over Rails.

However, as ErlyWeb is becoming more mature, my curiosity has been growing. With the recent releases of Rails 2.0 and Erlang/OTP R12B, I finally decided to sacrifice the better part of my weekend to give both frameworks some serious stress testing and see how they compare.

Before I could run the benchmarks, I had to figure out the physical setup. I needed at least two powerful servers with a fast link between them, and I didn't have this kind of hardware lying around. All I have is my MacBook, but running the benchmarks on my MacBook wouldn't prove much because it would be impossible to isolate the impact the clients would have on the servers by running on the same machine.

Thankfully, Amazon EC2 made this easy to solve. I could just fire up two EC2 instances and have one of them stress test the other over EC2's fast internal network.

Benchmarking can be complex. How do you benchmark a web app? There are so many moving pieces and possible user interactions. I decided to keep it simple: I would test the performance of rendering a single page displaying a simple dynamically generated table. There would be no database queries -- the table's contents would be hardcoded in the controllers and the views would programatically render the contents for each request.

I decided against using a database because I wanted to isolate the performance difference between ErlyWeb and Rails. 3rd party dependencies that may affect the results by introducing bottlenecks that would make one or both frameworks appear artificially slow. In addition, in many, if not most, webapps, the majority of requests can be served from the cache -- which means they don't trigger database queries -- so this test scenario is actually quite realistic.

The Test App

Here's a screenshot of the awesome test page I created:



You can see the generated page here (the output is identical for both apps except for minor whitespace differences).

This is the Rails controller code (songs_controller.rb):


class SongsController < ApplicationController
def index
@songs =
["Sgt. Pepper's Lonely Hearts Club Band",
"With a Little Help from My Friends",
"Lucy in the Sky with Diamonds",
"Getting Better",
"Fixing a Hole",
"She's Leaving Home",
"Being for the Benefit of Mr. Kite!",
"Within You Without You",
"When I'm Sixty-Four",
"Lovely Rita",
"Good Morning Good Morning",
"Sgt. Pepper's Lonely Hearts Club Band (Reprise)",
"A Day in the Life"]
end
end


This is the Rails view code (songs/index.html.erb):

Songs

<% for song in @songs %>



<% end %>

<%= song %>

This is the ErlyWeb controller code (songs_controller.erl):


-module(songs_controller).
-export([index/1]).

index(_A) ->
{data,
[<<"Sgt. Pepper's Lonely Hearts Club Band">>,
<<"With a Little Help from My Friends">>,
<<"Lucy in the Sky with Diamonds">>,
<<"Getting Better">>,
<<"Fixing a Hole">>,
<<"She's Leaving Home">>,
<<"Being for the Benefit of Mr. Kite!">>,
<<"Within You Without You">>,
<<"When I'm Sixty-Four">>,
<<"Lovely Rita">>,
<<"Good Morning Good Morning">>,
<<"Sgt. Pepper's Lonely Hearts Club Band (Reprise)">>,
<<"A Day in the Life">>]}.


This is the ErlyWeb view code (songs_view.et):


<%@ index(Songs) %>

Songs

<% [song(S) || S <- Songs] %>


<%@ song(S) %><% S %>

Setup

This is what I installed on the EC2 image (I started with the basic public Fedora Core 4 image):

- Erlang/OTP R12B.
- Yaws 1.73 (compiled with HiPE).
- ErlyWeb from trunk (compiled with HiPE).
- Rails 2.0.1
- Mongrel 1.1.1
- MySQL 4.1 (unused)
- Tsung 1.2.1.

Notes:

- Rails insists on connecting to a database even if you don't want or need one. There must be some hacky way of running Rails without a database, but it was easier for me to just setup MySQL so Rails stops complaining.
- I also compiled the ErlyWeb app with HiPE.

Tsung (http://tsung.erlang-projects.org/) is the tool I used for the stress testing. It's pretty simple to use -- you specify in an XML file where the server, what page(s) to request and the frequency of initiating new requests and then it runs the test and produces a report. You can read about it more on its website.

One interesting fact about Tsung is that it is written in Erlang to take advantage of Erlang's support for massive concurrency (there will be more on that later :) ).

I installed Tsung on the same machine image as the web servers because although during the test one instance runs the servers and the other runs Tsung, I wanted to have two identical EC2 machine images to ease setting things up.

During the test, I had both servers running on the server instance listening on different ports (Mongrel on port 3000, Yaws on port 3001). I then ran Tsung twice: first against Mongrel and then against Yaws. The tests did not overlap.

For the test, I configured Tsung to go through a sequence of 11 10-second phases during which Tsung would send requests to the server at increasing frequencies. In the first phase, Tsung would send 2 requests per second. It would then progress to 10, 20, 40, 100, 200, 400, 1000, 2000, 4000, and finally 10000 requests per second.

Caveat Emptor

Before I show the results, I would like to say that you should take them with a grain of salt. Many benchmarks have flaws. Maybe I messed something up during the installation. Maybe my methodology was wrong. Maybe I didn't perform some Mongrel optimization trick that boosts its performance. Maybe your application would behave quite differently from my test app. You have been warned.

Results

UPDATE: the initial results were inaccurate because I didn't run Mongrel in production mode. I updated the results and the graphs to reflect Rails's performance in production mode.

Below is the summary of the most important measurements.

Peak Performance
Rails: 15.1 112.8 requests/sec
ErlyWeb: 699.8 requests/sec



Peak Send Rate
Rails: 181.11 1360.78 Kb/sec Kb/sec
ErlyWeb: 5997.50 Kb/sec (hmm... did we hit EC2's limit?)



Total Size Sent
Rails: 1.65 5.83 MB
ErlyWeb: 28.47 MB



Peak Receive Rate
Rails: 84.87 175.79 Kb/sec
ErlyWeb: 941.84 Kb/sec



Total Size Received
Rails: 0.21 0.64 MB
ErlyWeb: 4.43 MB



Performance Degradation Point
(The phase at which the framework started sharply dropping the response rate)

Rails: 100 400 requests/sec phase.
ErlyWeb: 4000 requests/sec phase.



Note: these are the updated reports after running Mongrel in production mode

You can view the full Tsung reports here: ErlyWeb Rails

These reports are from the second test (30 second sustaining at 10 requests/sec):
ErlyWeb Rails

You can also download the reports, zipped: reports.tar.gz

Conclusion

ErlyWeb's peak response rate was 47x 6x Rails's on similar hardware.

I'm actually surprised by the results. I expected ErlyWeb to win, but I didn't expect ErlyWeb to outperform Rails by such a wide margin.

The results probably reflect differences in language and runtime more so than implementation details in the frameworks. After 20 years of refinement driven by needs of high performance telcom applications, Erlang delivers.

Update:
Looking at the peak send rate for ErlyWeb -- an almost round 6 MB/s, I wonder if ErlyWeb just saturated EC2's internal bandwidth, and whether ErlyWeb could perform even better on a faster link.

Update 2:
I ran another test, this time hitting each server at 10 requests/sec sustaining for 30 seconds to measure average response time. Rails's response time was in the 120-180 5-7 msecs range, and ErlyWeb's was in the 1.4-1.8 msec range. This means that under medium to light load, ErlyWeb serves the 'songs' page roughly 100x 4x faster than Rails.

Vimagi Speedups

Vimagi was much slower than it should have been.

I had foolishly compiled Vimagi in production with {auto_compile, true}. This option tells ErlyWeb to scan the app's source files and recompile all the ones that have changed since the last request. This feature greatly speeds up development because you can edit your file, reload the page, and immediately see the effects of your changes. It was also convenient for me in production because after checking in some changes from my dev box I would just 'svn up' on the production server and the changes will be deployed automatically. Unfortunately, I didn't realize what a negative impact it has on performance. Thankfully, David King brought this to my attention on the ErlyWeb mailing list and I've since disabled auto_compile on the production server.

Vimagi performs much better now. Clicking around the site, most pages now load in 0.3 - 0.5 secs according to YSlow. It feels much faster. In fact, Vimagi's performance is now similar to BeerRiot's, which is no small feat.

By the way, I'm not sure about BeerRiot, but Vimagi's pages are all dynamically generated -- I haven't implemented any caching (my VPS can easily handle the current traffic levels, so implmeneting caching right now would be a premature optimization). For an entirely dynamic site, I think this is very good performance.

Erlang Quote of the Day

Who cares if Erlang starts slowly - it was designed to start once and never stop - we have systems that have run for 5 years - a two seconds start-up time amortized over 5 years is not *too* bad.

- Joe Armstrong

(From http://www.nabble.com/idea%3A-service-pack-one-tf4800752.html#a13735173)

ErlyWeb Presentation, Dec. 6th at Berkeley, CA

The BayFP group is on a roll: 3 presentations on functional web development frameworks in 3 months. Alex Jacobson presented HAppS, David Pollak presented Lift, and in the next meeting I'll be presenting ErlyWeb.

The meeting will take place on Dec. 6th in Berkeley. You can find directions here.

There will be a $300 attendance fee.

Just kidding -- it's free. Please attend.

BayFP Meeting Tomorrow

I'm going to the BayFP meeting tomorrow. David Pollack will present lift, the web framework he created for Scala. Anyone who's interested is welcome to attend.

Wide Finder Project Summary

WF II: Erlang Blues (9/22)

Dear Erlang I: · I like you. Really, I do. But until you can read lines of text out of a file and do basic pattern-matching against them acceptably fast (which most people would say is faster than Ruby), you’re stuck in a niche; you’re a thought experiment and a consciousness-raiser and an engineering showpiece, but you’re not a general-purpose tool. Sorry.

WF XI: Results (10/30)

Erlang 6.46
Ruby 50.16
...
Analysis · Are you kidding me!?!? Getouttahere. Maybe someday.

Questioning Questioning Functional Programming

I spotted today on reddit an article from O'Reilly Radar titled Questioning Functional Programming. It's based on a programming contest in which more teams used non-functional languages than functional ones.

[The competition is] notable for two things: the fiendish complexity of the problem and the absence of functional programming languages in the final results. Top place went to a C++ program, with a Perl team right behind in second. In third place was "Team Celestial Badger" using OCaml and C++. In all there were 81 teams using C++, 67 using C, 66 using Haskell, 64 using Python, 52 using OCaml, 48 with Java, and 35 with Perl. Only 4 tried Erlang (the same number as used Delphi).

According to the report from the competition:

...As far as functional programming is concerned, we must conclude that functional languages didn’t fare too well this year (although in the Top 15 there were five users of OCaml and three of Haskell). Better luck next year!

There were 421 teams, of which 122 used functional languages -- about 29% (which ought to be much higher than their rate of adoption by the industry). Still, over 50% of the top 15 places were won by the teams that used functional languages. It looks like good performance to me.

It says to me that we're still in the early days of widespread adoption of functional programming.

I don't think this is what it says. Functional programming languages are obviously less popular than imperative ones. Maybe functional languages will never be as widespread as imperative languages. However, this competition is just a competition. I doubt it says much about language adoption by the industry now or in the future. (By the way, has anyone heard of Ruby? :) ) If anything, this competition suggests that functional programming languages are quite widespread, given their high rate of adoption by the contestants.

Like Ruby, which had tiny uptake until Rails and 37 Signals boosted it into the stratosphere, functional programming languages lack a reason to use them.

If Ruby lacked a reason to be used before 37 Signals, why did DHH use it to create Rails? He could have stuck with PHP. The reason to use Ruby did exist, even if fewer people knew about it.

"Functional programming languages lack a reason to use them." I don't think that great concurrency, scalability, fault tolerance, distributed programming, hot code swapping, Mnesia, Yaws, and the rest are useless.

(In fact, Amazon has made Erlang much less useless today by announcing the new 4-core and 8-core options for EC2 instance. I can't wait to get my hands on those :) )

Even those who advocate them [functional languages] aren't comfortable enough with them to push on with them in the face of hard problems.

The teams that were comfortable with functional languages did push on them in the face of hard problems: those teams used functional languages in the contest. They took 8 of the top 15 places, too.

I must say that I disagree with the conclusions of the article.

I'll end this posting with a relevant message from the Erlang-questions mailing list yesterday:

> Hi.
> Recently, I have been trying to learn a language other than Java.
> Erlang seems to be ready for the prime time (used in the commercial world).
> On the other side, OCaml seems to be a cool and capable language.
> I want to learn something useful and fun.
> Tell what do you think ?
> (if you are Erlang die-hard, please put this aside ;))
>

When I was evaluating Python alternatives for building the core
technology behind MochiAds I tried out a bunch of languages and Erlang
was the only one that was easy for me to learn and had the right
balance of features, performance, and reliability. A year later we
have about 16 machines running 80 Erlang nodes powering about 16
different "components" of our infrastructure and 4 people working on
it at the moment (originally it was just me). It worked out so well
that we rewrote the server component of our MochiBot service in Erlang
and we've been using it to build lots of internal tools such as our
monitoring software, our single sign-on service, etc. as well. None of
us had previous Erlang experience, but we're all very comfortable with
it now.

After about a year with Erlang, I'm not sure I could part with hot
code loading, light-weight processes, and multiplexed socket IO for
writing servers. Also, Mnesia has been really useful to us to
temporarily store "real-time" data (ram_copies) so that we don't have
to make users wait for it to get batched into the SQL databases. The
distribution stuff mostly Just Works once you figure out how to set it
up (though we did have one bad experience with a network partition due
to a switch acting up, it was recoverable manually).

O'Caml is a useful language too, but for writing a network app I can't
really imagine going with anything but Erlang if you're looking for
redundancy and scale. Unless you want to write your own half-baked
Erlang-like system before even trying to solve something a little
closer to your actual problem domain.

-bob

It looks like functional programming must be good for something.

Blog Maintenance

I moved my blog to a different hosting provider. Please let me know if you notice any URLs that the move may have broken.

I also removed that annoying CAPTCHA plugin so you shouldn't have problems leaving comments from now on. If the plugin prevented you from leaving comments in the past you can now go ahead and leave them now. (I may have to put CAPTCHA back if Akismet flakes out on me but I hope I won't have to resort to that.)

I apologize for any inconvenience.

More Vimagi Goodies

- You can now undo strokes.
- There's a better tools interface.
- You can embed paintings on any site, and even play them back!

For example, here's a great painting by Rivka. It's titled "horror flicks".



Enjoy!

If I Were Building Amazon.com...

I would use ErlyWeb :)

From Amazon's Dynamo by Werner Vogel:

"In Amazon’s decentralized service oriented infrastructure, SLAs play an important role. For example a page request to one of the e-commerce sites typically requires the rendering engine to construct its response by sending requests to over 150 services. "

Unless all 150 service requests are executed sequentially, which is hard to believe, Amazon's page rendering ought to be able to benefit from Erlang's lightweight concurrency -- even more so than Vimagi :)

This makes me think about adding the following return value to ErlyWeb controller function:


{concurrent, [{ewc,...}, {ewc, ...}, {ewc, ...}]}


This would tell ErlyWeb to render each component in a different process. It may not make a big performance difference in rendering simple pages (it might even slow it down a bit), but for some applications (the future Erlmazon? :) ), it could be useful.

What do you think?

Vimagi Updates

Vimagi news:

- Alpha channel support. You can now pick the transparency level from the color box.
- Clearing a painting erases all previous stroke data. From now on, the stuff you do before clearing the painting doesn't get played back.
- Performance improvements.
- Nightly backups to S3: your precious paintings are safe :)

Enjoy!

ErlyWeb is Facebook-Ready

In case you didn't know, Bryan Fink of BeerRiot fame has created a Facebook library for Erlang called erlang2facebook. You can get it from http://code.google.com/p/erlang2facebook/. It is bundled with a complete Hello World Facebook app that is built with ErlyWeb, so you can hit the ground running with your Erlang Facebook development.

I can think of some fun projects to build with this library. If you're going to use it to build an app, please let me know!

Embedding Vimagi

Dear reader, I have good news. The internet is finished. The missing link was finally implemented.

You can now embed your Vimagi paintstream in any web page :)

All you have to do is add the following to your HTML, replacing [USERNAME] with your Vimagi username:

My Vimagi PaintStream

Enjoy!

Great Concurrency -> Better Webapps

I've been asked a few times: Is concurrent programming really useful for web developers? Web servers already handle client requests in different threads/processes, so application code doesn't need to use concurrency explicitly, right?

My opinion is that althought it's possible to write many database-driven web apps without using concurrency, concurrency can still be useful.

Vimagi has some real-time features in that rely heavily on concurrency, but it also helps in a more common scenario: speeding up page loads.

In Vimagi, every time someone views a painting or a user profile, a SQL query is executed to increment the view counter for that page. This is more-or-less what the controller code does:


show(A, Id) ->
painting:increment([num_views], {id,'=',Id}),
Painting = find_id(Id),
{data, Painting}.


This caused a problem. When traffic increased, the 'increment' function, which executes an UPDATE statement in the database, started performing poorly, thereby noticeably slowing down page loads. Occasionally, it even timed out after 5 seconds, showing an error screen to the user.

I increased the timeout to 20 seconds. This helped avoid the error screen, but the response time was still too long.

Erlang made this almost too easy to fix. All I had to do was change


painting:increment([num_views], {id,'=',Id})


to


spawn(
fun()->
painting:increment([num_views], {id,'=',Id})
end)


It's darn simple, and it works. Instead of executing the UPDATE statement in the process that's responsible for rendering the page, I spawn a new process and execute the transaction there. Spawning processes in Erlang is dirt-cheap, so I don't have to worry about performance or memory issues. The user gets a quick response, the view counter is incremented, and the problem goes away.

Introducing Vimagi.com

I just launched Vimagi, my first ErlyWeb app: vimagi.com.

Painting is fun. Sharing paintings with your friends is also fun. Painting together with your friends is a new kind of fun you can have on Vimagi.

Give it a try and let me know what you think -- I'll appreciate the feedback.

Note: Vimagi is in early beta. It probably has many bugs. Please email me if you find any.

Fancy Language/Framework Poll Results

My Wufoo poll has reached 99 entries! Let's check out the results.

Most popular languages:
Ruby: 31%
Erlang: 23%
Python: 21%
Lisp: 4%

Most popular frameworks:
Ruby on Rails: 39%
All web frameworks suck: 16%
Django: 15%
ErlyWeb: 6%

You can see the report here, with pretty graphs: http://yariv.wufoo.com/reports/a-fancy-languageframework-report/.

(Btw, it would be great if Wufoo allowed embedding public reports on other pages.)

Ruby/Ruby on Rails is the most popular language/framework among my blog readers. This is interesting, given that I don't write about Ruby much. My take on it is that many Rubyists read my blog because a) they are numerous (I'm sure there are many more Ruby hackers than Erlang hackers) and b) they are curious about other languages, especially Erlang. The same goes for Pythonistas.

It's interesting how many people think that most web frameworks suck. Maybe they don't do web programming (but then, why would they think tools they never use suck?), they love the PHP/JSP model (somehow, I doubt it), or they have better solutions for their needs. It's a mystery.

A couple of responders hate all computer languages. I wonder why they are reading my blog, then. The self-help section is yet to be launched... :)

A curious response is by a Lisp hacker who likes ASP.NET.

Only a couple of responders like Java. It's strange, given that Java is one of the most widespread languages. Maybe I can make ErlyWeb more marketable by renaming it 'Erlang Enterprise ConcurrentBean Application Server' :)

This is too much fun.

A Fancy Language/Web Framework Poll

Making forms with wufoo is so darn easy I just had to create one. It is below, in all its glory: a language/web framework poll for my blog readers.


Powered by Wufoo

Thanks for participating. Your input is very valuable -- I think.

SF Bay Area ErlLounge, Aug 15

In case you haven't seen the announcement on the mailing list (http://www.erlang.org/pipermail/erlang-questions/2007-August/028324.html), the 2nd SF Bay Area ErlLounge will take place on Aug 15. Anyone who's interested is welcome to attend.

BeerRiot - The First ErlyWeb App

Many people may not know this, but ErlyWeb has an app.

It's called BeerRiot (http://beerriot.com), and its creator is Bryan Fink, whose been brewing it for a few months now (pun intended). He blogs about it at http://blog.beerriot.com/. (My favorite quote: "To those who are considering Erlang and ErlyWeb as a framework for their website: just go for it.")

BeerRiot is the quintessential web destination for beer aficionados. It has a beer database, user-comments, votes, and tags. It even comes in black and white background flavors. The only feature it lacks is letting you taste the beers somehow -- I guess I'll have to add this capability to the ErlyWeb todo list :)

According to my highly unscientific and subjective performance tests, BeerRiot feels very fast. I measured page load time with YSlow, which reported just 0.2 seconds for the homepage.

Kudos to Bryan for making such a nice app, and for being the first person to create a production ErlyWeb app.

Erlang-Style Concurrency in Google Gears

I heard about Google Gears when it came out a few weeks ago, but only a couple of days I took it for a short test drive. I was surprised to see that Google Gears uses Erlang-style concurrency (well, sort of).

Google Gears supports asynchronous background processing using the WorkerPool module, which lets your Google Gears app execute multiple tasks concurrently without blocking the main UI thread. What I find interesting thing about WorkerPool is that threads only communicate by message passing and that there is no API for synchronizing access to objects that are passed around between threads. This is a departure from the traditional concurrency toolbox in imperative languages -- using synchronized access to shared memory.

It's so easy to use it's almost... Erlangy :)

So how does Google Gears pulls off this feat of simplicity?

Google Gears only lets you send strings between threads. Since Javascript strings are immutable, there's no danger in letting multiple threads share them.

This solution is clever, but it comes at a cost -- you must serialize your objects before sending them between threads, which incurs a performance penalty (some extra coding, too). I don't expect the performance penalty to noticeably impact most Google Gears apps, though, which are mostly offline adaptations of standard webapps.

Google Gears has managed to pull off easy concurrency in an imperative language by relying on the immutability of strings in Javascript, but this approach is only practical on the desktop. In a server app with higher performance and scalability requirements, sending serialized objects would probably slow things down too much.

All in all, I think Google Gears is a great solution for adding offline support to webapps, and it gets bonus points for using Erlang-style concurrency. Now I'm just waiting for Safari support :)

San Francisco ErlLounge, Jul 18th

About a month ago, I packed my stuff, said farewell to Boston, and moved to the San Francisco Bay Area. I currently live in the town of San Mateo, where occasionally cars drive by and people walk their dogs. It's very exciting. I will soon move to Palo Alto for a few months, and then who knows. Whatever it is, it won't be as lively as San Mateo.

I'm planning on attending the San Francisco ErlLounge on Jul 18th. Visit http://umichigan.facebook.com/p.php?i=2223888&k=33553ba7bd for details. You should attend if you can. I'll tell you thrilling stories about San Mateo. I promise.

On an unrelated note, the spammers got me beat, and I finally gave up and disabled comments on this blog. If you want to share your thoughts, you can email me at yarivsblog at gmail dot com.

Concurrency Buzz

There's been quite a buzz lately in the programming blogsphere about concurrent programming. I think the software world is paying increasing attention to concurrent programming (and Erlang) due to the following reasons:

- The apparent decline of Moor's law coupled with the rise of multi-core CPUs.
- The "traditional" (i.e. shared-memory based concurrency) employed by most languages is flawed: it's too hard to grasp, it's too error-prone, and it's not multi-core friendly.
- Joe Armstrong's book, Programming Erlang has made Erlang accessible to a wide audience.
- Reddit.

There's more to Erlang style concurrency than lightweight processes and message passing. I don't expect other (popular) languages to fully gain Erlang-style concurrency (not soon, at least) because to do so they would have to adopt the following combination of traits:

- A scheduler that uses a mixture of lightweight processes and system threads (SMP Erlang spawns a system thread per CPU).
- Immutable variables.
- General avoidance of long blocking calls to native code in libraries.
- A garbage collector that utilizes per-process heaps to avoid freezing the entire VM during sweeps.
- A revised approach to error handling based on supervision trees and non-defensive coding.
- Web servers, database drivers, libraries, tools, etc. that actually use the new concurrency primitives.

As the saying goes, never say "never"... :)

Does that mean that you need Erlang to make your applications utilize available CPUs on multi-core machines? Of course not. However, when 80 core machines start appearing in our data centers in a few years, at least some people will take comfort in Joe Armstrong's statement:

"Your Erlang program should just run N times faster on an N core processor"

SlideAware: From Python to Ruby to Erlang

SlideAware has created a slick web-based solution for managing the life cycle of Powerpoint presentations. They started using Python, then they switched to Rails, and finally settled on pure Erlang. Using Erlang, they replaced a combination of Lighttpd + RoR + SQLlite + XMLRpc + Jython + Lucene with the much simpler and highly scalable stack of Yaws + Mnesia + Erlang.

They wrote an interesting article describing their technology choices. Check it out at http://slideaware.typepad.com/slideaware/2007/04/from_python_to_.html.

Announcing ErlyWeb 0.6

I'm pleased to announce the release of ErlyWeb 0.6. This very exciting release contains a new Mnesia driver for ErlyDB created by Matthew Pflueger and a Postgres driver (based on the open source driver from Erlang Consulting) created by Roberto Saccon. Finally, you can use a single database abstraction layer in Erlang to access MySQL, Postrgres and Mnesia without writing a single line of SQL or DBMS-specific logic :)

You can get ErlyWeb 0.6 from the ErlyWeb website or from its Google Code repository. Make sure to check out CHANGELOG.txt for a complete list of changes.

If you have any questions regarding the new drivers, please ask the ErlyWeb Google Group.

An Erlang Pipe

Yahoo Pipes is one of the most innovative apps I've seen: it lets you program data feeds. More specifically, you can use Pipes build programs (called "pipes") that generate output feeds from input feeds by combining, filtering, and manipulating the inputs using different modules from the Pipes IDE. The IDE has a friendly graphical interface, letting you drag, drop and connect different components easily. The IDE is implemented in Javascript, which is impressive.

I couldn't resist playing with Pipes, so I created an Erlang pipe. The Erlang pipe reads the feeds from programming.reddit.com and lambda-the-ultimate.org, combines them, and filters the results by accepting only articles whose contents contain the word 'Erlang'. It then combines the result with the inputs from planeterlang.org.

Check out this pipe at http://pipes.yahoo.com/pipes/pipe.info?_id=4jVpGgjt2xGttygedrq02Q.

ErlyWeb Cheat Sheet

Brian Olsen has created a very helpful cheat sheet for ErlyWeb. Check it out at http://www.pushon.org/erlyweb/cheat_sheet.html.

Boston BarCamp2

Is anybody who reads this blog going?

http://barcamp.org/BarCampBoston2

I think I may go and try to talk some people into using Erlang in their startups.

Update (2 hours later):

It looks like my job is going to be easier than expected. I checked out the at the BarCamp real time visual aggregator (http://barcamp.org/BarCampNewsVisualAggregator), and guess what I saw?

TECH. PROBLEMS

This experiment may cause to much traffic for pbwiki... to be improved...

Maybe yaws could be a better platform for serving the BarCamp aggregator: http://www.sics.se/~joe/apachevsyaws.html :)

Announcing ErlyWeb 0.5

I'm pleased to announce that ErlyWeb 0.5, the biggest and best release yet, is now available to download from erlyweb.org. There's a long list of improvements and fixes, of which you can read here.

As I indicated in a previous posting, ErlyWeb no longer supports app views. If you are upgrading from an earlier version, you should convert your app view to an html container and use the 'phased' return value from the app controller, as described in the updated documentation on the website.

ErlyWeb is still in alpha, and will likely remain this way until somebody has used it to build a complete production app. So, who will reach the finish line first? :)

Web applications often have pages in which the user can edit the fields of an existing database record and then save the updated record by submitting the form. To generate such an 'edit' page, you generally have to read the record from the database, generate an HTML form whose fields are pre-populated with the record's data, and then send the form back to the browser. Let's look at a few ways of doing this with ErlyWeb (for instructional purposes, we won't be using "-erlyweb_magic(on)." :) ).

Let's suppose we want to provide a form for editing the records of a database table called 'language'. We can create an ErlyDB module for working with this table as follows:

language.erl:


-module(language).


When we call erlyweb:compile() (or when ErlyWeb auto compiles our modules becase we have turned 'auto_compile' on, as one should do during development), ErlyDB reads the metadata for the 'language' database table and generates a large number of functions in the 'language' module covering the whole exciting CRUD gamut.

Now let's create a basic controller implementing the data access logic for the 'language edit' page. I will only show the most stripped-down logic necessary for rendering the 'edit' form, which is the focus of this tutorial.

language_controller.erl:

-module(language_controller).
-export([edit/2]).

edit(A, Id) ->
case language:find_id(Id) of
undefined ->
{data, {error, no_such_record}};
Language ->
{data, Language}
end.


When ErlyWeb receives a request such as "http://myapp.com/language/edit/1", it will invoke the 'edit/2' function in language_controller. This function will look up the database record using a SQL query such as "SELECT * FROM language WHERE id=1", and send the resulting record directly to the view function, or the tuple {error, no_such_record} if no record matches the requested id.

Now let's take a first pass at making a the view for this component (note that in this tutorial, we'll avoid any fancy automatic form-generation methods and write most of the HTML by hand).

language_view.et:

<%@ edit({error, no_such_record}) %>

ouch, the record doesn't exist

<%@ edit(Language) %>

name:

paradigm:

....




Looks good so far? Well, it's not really supposed to, because this code has some problems.

The most urgent problem is that this code doesn't check that the record's fields are formatted as iolists. If any of the record's values is 'undefined', for example, this code will cause the process to crash (by 'crash' I don't mean the catastrophic definition of 'crash' that is used with most programming languages, but the Erlang definition, which means the lightweight process in which the crash occurs has died -- an isolated event inside the VM, nothing more).

We can address this issue in a few different ways. I'll show you one simple way, in which we create a function in the view module to do the proper conversion. Using this method, this would be the modified code for the new language.et:


<%@ edit(Language) %>

name:

paradigm:

....



<%@ show(Language, Field) %>
<% erlydb_base:field_to_iolist(language:Field(Language)) %>


Note that this code uses the Erlang way of doing dynamic function dispatch: you use a variable in place of the module's function (and/or the module itself) you want to call. This is an extremely useful feature -- know it and it will serve you well.

The code works, but it irks me somewhat. The issue is that I don't want the view logic to know anything about ErlyDB records and how to access and format their fields. Views are supposed to be simple, and should only use the most rudimentary logic required to render their parameters. Putting ErlyDB logic in the view violates this rule.

So, how do we fix this issue? We have a few options. One option is for the controller to pass the view a list of pre-rendered fields instead of the Language record itself. This could be done in the controller by returning the following:


{data, [erlydb_base:field_to_iolist(language:Field(Language)) ||
Field <- language:db_field_names()]}


To handle this list, the view function would have to change, too:


<%@ edit([Name, Paradigm, ....]) %>


name:

paradigm:

....


Using this approach, we've succeeded at stripping the view function of ErlyDB-related logic, but there is a downside: we now have to maintain as the function parameter a list of field variables that matches exactly the length and order of the field list returned from language:db_field_names(). This is acceptable for records whose field list is small and doesn't change much, but for records with large numbers of fields this quickly becomes a pain to maintain.

Our search for the ideal solution isn't done, but from my experience with Erlang in specific and functional programming in general, the ideal solution -- elegant, clean, flexible -- almost always exists, even if you haven't found it yet :)

What we ultimately want is to be able to look up properly formatted field values by name without worrying about the fields' order and without calling ErlyDB functions (or functions generated by ErlyDB) directly. To do this, we can go the "traditional" (i.e. lame :) ) route of passing the view function a data structure mapping field name keys to data values. In Erlang, this could be a simple property list, a dict (i.e. a hash table), a gb_trees structure, or some other associative container. With this approach, the view function would populate the field values using snippets such as


<% dict:find(name, LanguageDict) %>


However, I don't like this approach because it incurs the overhead of creating and using the associative container (in a real app, this overhead would probably be negligible, but I still don't like it :) ), and it's tedious to repeatedly call the associative container functions to look up field values. In addition, real purists would argue that the view function shouldn't know the type of the container, or even that we're using some associative container to look up the values of our fields in the first place!

FP to the rescue.

My favorite solution to our view dilemma is to change the controller's 'edit' function as follows:


edit(A, Id) ->
case language:find_id(Id) of
undefined ->
{data, {error, no_such_record}};
Language ->
{data,
fun(Field) ->
erlydb_base:field_to_iolist(language:Field(Language))
end}
end.


What's going on here? Instead of passing the view function some data structure, we pass it a closure, i.e. a function whose logic uses one or more variables that exist in the closure's lexical scope. Using this closure, we can get the properly formatted value for any field of the Language record with minimal performance -- and coding -- overhead. Here's how we use it in the view function:


<%@ edit(F) %>

name:

paradigm:

....


As you can see, using closures in this scenario gives us the best of all worlds: almost no performance overhead, minimal code, no maintenance of field lists, and clean separation between controller and view logic.

In summary, if you want to keep your code clean, simple, and properly decoupled, closures are often your best friends.

Introducing... The ErlyWeb Logo!

My plea for help in the graphics design department in the making of an ErlyWeb logo has yielded a few candidates, and although I sincerely appreciate the efforts of the people who made them, and they gave me some good ideas, none of them has truly captured my fancy. So, I decided to take matters into my own hands and finally create a real logo for ErlyWeb.

Without further ado, I hereby present ErlyWeb's shiny new logo:





(This is the big version. A small version is on http://erlyweb.org.)

This logo is inspired by Erlang, Big Sur and Lisa Simpson.

(When I wrote that my graphic design skills were non-existent, you didn't think it really meant there was a Picasso inside me trying to break out, did you? :-) )

ErlyWeb Tutorial: Using Components To Create Dynamic Containers

ErlyWeb's app views are a basic mechanism for sharing common view elements between components. An app view is a simple Erlang module or ErlTL template file that nests pre-rendered components in static (generally HTML) content. Most simple applications probably have an app view that looks like this:







<% Data %>




This mechanism works well when your components share only static view elements. But what do you do when your components share dynamic view elements? If this is the case, you have a few options:

- You could put the logic for generating the dynamic content in the view module. However, this would be ugly, and it would go against one of ErlyWeb's primary purposes, which is to help you create a clean separation between application logic and view logic. Let's forget this option and move on.

- If all your components share an identical set of dynamic elements, you can change your app controller's hook function, so instead of returning '{ewc, A}', it returns a list containing the 'ewc' and/or 'data' tuples common to all components. For example, if all your pages have sidebar, your hook/1 function could return '[{ewc, sidebar, [A]}, {ewc, A}]'. This tells ErlyWeb render each component in the list, and then pass into the app view a 2 element list, wherein the first element is the rendered sidebar and the second element is the rendered component that corresponds to the browser's request. To properly handle this new parameter type, you could change your app view as follows:


<%? [Sidebar, Content] = Data %>

<% Sidebar %>

<% Content %>

Again, this works well when all your pages have the same sidebar component. However, what if different pages have different sidebar components? Or what if some pages have a sidebar and some don't? Let's look at the next option.

- Starting from ErlyWeb 0.4, your controllers can return a {response, Elems} tuple, whose Elems list may include the {app_view_param, Ewc} tuple (for more info on the 'response' tuple, check out the ErlyWeb docs. Similar to the first option, this response tells ErlyWeb to render the (list of) 'ewc' and/or 'data' tuples contained in the Ewc variable, and then pass into the app view a 2-element list containing the rendered components followed by the rendered content for the request. In addition, you can use the {app_view, ModuleName} tuple to pick a specific app view (or disable the app view entirely) instead of using the default one.

This approach works, and it offers greater conveniece in customizing the dynamic app view data for each component (or each component function), but I don't like it. The reason is that I don't think that components should know or control what happens in the areas of the page outside of their own rendered HTML. In addition, it gets confusing if multiple nested sub-components define the 'app_view_param' element in their responses: which one should ErlyWeb be use? Given these shortcomings, I'm considering deprecating if not outright removing support for the 'app_view' and 'app_view_param' tuples from the 'response' tuple in ErlyWeb (it is and an experimental feature, after all, as the documentation notes). So, let's look at the next option.

- The last, and arguably best approach, is to use ErlyWeb's component system to create dynamic containers. A container is a component that contains one or more sub-components, which are given to the container as parameters. The container tells ErlyWeb to render its parameters together with any dynamic data specific to the container. For example, to create a container that renders its sub-component(s) together with a sidebar, you could use the following code:

sidebar_container_controller.erl:

-module(sidebar_container_controller).
-compile(export_all).

%% This tells ErlyWeb to reject browser requests that try to
%% access this component directly
private() -> true.

index(A, Ewc) ->
[{ewc, sidebar, A}, Ewc].


sidebar_container_view.et:

<% Data %>

<%@ index([Sidebar, Contents]) %>







<% Sidebar %>

<% Contents %>




Using this container, any component can render a subcomponent with a sidebar by simply returning the tuple '{ewc, sidebar_container, [A, {ewc, subcomponent, [A]}]}'. What I like about this container, beside its flexibility and reusability, is that it doesn't require any special logic nor does it rely on any features beyond ErlyWeb's simple and elegant component model.

An ideal place to use this container is in the app controller's hook function. For example, to achieve the same effect as the app view-based approach from the second option, you could implement this function as follows:


hook(A) ->
Ewc = erlyweb:get_initial_ewc({ewc, A}, myapp_erlyweb_data),
{ewc, sidebar_container, [A, Ewc]}.


(The first line may look a bit cryptic -- indeed, it relies on a function that isn't included yet in the official distribution -- but just know that it enforces the 'private' policies on component requests that come from clients.)

A nice thing about this approach is that it helps simplify the app view, which doesn't require unpacking the Data parameter anymore and laying out the sidebar and the contents as the second example illustrates.

We could take this a step further and make the app view obsolete by creating an HTML page container as follows:

html_container_controller.erl:

-module(html_container_controller).
-compile(export_all).

private() -> true.

index(A, Ewc) ->
Ewc.


html_container_view.et:

<% Data %>

<%@ index(Data) %>





<% Data %>




Now, we could implement hook/1 as follows:


hook(A) ->
Ewc = erlyweb:get_initial_ewc({ewc, A}, myapp_erlyweb_data),
{ewc, html_container,
[A, {ewc, sidebar_container, [A, Ewc]}]}.


With this container structure, out app view becomes


<% Data %>


As you can see, using the container approach, we've replaced the somewhat crippled app view with a much more flexible and powerful ErlyWeb component. Not bad, huh?

So where does this lead us? Will app views soon be a thing of the past? I think this may very well happen :)

ErlyWeb Tutorial: Creating a Simple Login Page

Let's see how to create a simple login component with ErlyWeb.

Update: I simplified the HTML to reduce clutter.

First, let's take a look at login_view.et:


<% Data %>

<%@ index({Username, Errs}) %>
<% [error(Err) || Err <- Errs] %>


login

username:

password:

<%@ error(Err) %>

- <% error_msg(Err) %>

<%@ error_msg({value_too_short, FieldName, Min}) %>The <% FieldName %> must be at least <% Min %> characters.
<%@ error_msg(invalid_credentials) %>Invalid username/password.


As you can see, the 'index' function of login_view.et accepts a 2 element tuple. The first element is the value of the 'username' field, and the second element contains a list of error values. An error value can be any Erlang data type. The error_msg/1 function maps error values to their textual message. This is done by simple pattern-matching (aside: damn, I love pattern-matching :) ).

Before we take a look at the controller, let's implement a nice little helper function that contains some boiler-plate code for form validation.

Update: The following is generic, reusable code that can work for any form validation, not just login pages. It may even be included in the framework at some point.


validate(A, Fields, Fun) ->
Params = yaws_api:parse_post(A),
lists:foldl(
fun(Field, {Vals, Errs}) ->
case proplists:lookup(Field, Params) of
none -> exit({missing_param, Field});
{_, Val} ->
Val1 = case Val of undefined -> ""; _ -> Val end,
Acc1 =
case Fun(Field, Val1) of
ok ->
{[Val1 | Vals], Errs};
Err ->
{[Val1 | Vals], [Err | Errs]}
end,
Acc1
end
end, {[], []}, lists:reverse(Fields)).


This function takes the Yaws arg record, a list of expected field names to validate, and a function object that takes 2 parameters -- the field name and the value -- and returns 'ok' or an error code. The validate/3 function returns a 2 element tuple. The first element is the list of values corresponding to the field names, and the second element is a list of error codes returned by the function object as it was applied to each field. validate/3 also converts empty field values from 'undefined' to empty strings so they can be passed to the view component, which always returns iolists, verbatim.

Now, let's take a look at login_controller.erl:


index(A) ->
case yaws_arg:method(A) of
'GET' ->
{data, {"", []}};
'POST' ->
case validate(A, ["username", "password"], fun validate_field/2) of
{[Username, Password], []} ->
case app_user:find(
{'and',
[{username,'=',Username},
{password,'=',crypto:sha(UserName ++ Password)}]}) of
[] ->
{data, {Username, [invalid_credentials]}};
[User] ->
Key = crypto:rand_bytes(20),
Encoded = http_base_64:encode(binary_to_list(Key)),
app_user:save(app_user:key(User, Key)),
{response,
[yaws_api:setcookie("key", Encoded),
{ewr, main}]}
end;
{[Username, _Password], Errors} ->
{data, {Username, Errors}}
end
end.

validate_field("username", Val) when length(Val) > 4 -> ok;
validate_field("username", _) -> {value_too_short, "username", "5"};
validate_field("password", Val) when length(Val) > 5 -> ok;
validate_field("password", _) -> {value_too_short, "password", "6"} .


If the request is a GET, login_controller passes into the view function an empty string for the username and no errors. The it's a POST, login_controller checks that the fields have passed basic validation, i.e. their values pass the minimum length requirements. If so, it checks that the username/password hash combination is in the database. If is it, login_controller assigns the user a random 20 byte key string as a cookie value, which it also saves in the database (in this example, we use the database for session management, but this isn't required) and then redirects to the 'main' component (note: this 'response' tuple is only available in ErlyWeb 0.4, which hasn't been released yet). If the username/password combination is invalid, login_controller passes to the view function the previous Username value and the 'invalid_credentials' error code.

This example illustrates a few cool things about Erlang/ErlyWeb:

- ErlyDB makes working with the underying RDBMS very simple and natural.
- Pattern matching is a joy, both in validation code and in ErlTL templates.
- ErlSQL lets us create short SQL statement snippets dynamically while avoiding manual iolist construction and guaranteeing protection against SQL injection attacks.
- Everything is so easy! You gotta love Erlang :)

Alright, now back to work.

My Talk at MSLUG

On January 17th, I'll be giving a talk about my adventures in open source Erlang at the Montreal Scheme/Lisp User Group (MSLUG) meeting. For more information, visit http://schemeway.dyndns.org/mslug.

I hear the members of this group have a strong interest in Erlang. Guillaume Germain, who wrote Termite, an implementation of Erlang style concurrency for Gambit-C Scheme, is a member, and so is Marc Feeley, the author of Gamit-C. I'm looking forward for my second opportunity to be surrounded by people who are interested in Erlang :)

This meeting is open to everyone, so if you're interested in attending, you're more than welcome.

Aside: I lived in Montreal for a year when I was 15. My last visit was about 5 years ago, so It'll be nice to visit again. It's been a while since I've had some good poutine! :)