Wednesday, October 31, 2007

Wide Finder Project Summary

WF II: Erlang Blues (9/22)


Dear Erlang I: · I like you. Really, I do. But until you can read lines of text out of a file and do basic pattern-matching against them acceptably fast (which most people would say is faster than Ruby), you’re stuck in a niche; you’re a thought experiment and a consciousness-raiser and an engineering showpiece, but you’re not a general-purpose tool. Sorry.


WF XI: Results (10/30)


Erlang 6.46
Ruby 50.16
...
Analysis · Are you kidding me!?!? Getouttahere. Maybe someday.

Tuesday, October 16, 2007

Questioning Questioning Functional Programming

I spotted today on reddit an article from O'Reilly Radar titled Questioning Functional Programming. It's based on a programming contest in which more teams used non-functional languages than functional ones.


[The competition is] notable for two things: the fiendish complexity of the problem and the absence of functional programming languages in the final results. Top place went to a C++ program, with a Perl team right behind in second. In third place was "Team Celestial Badger" using OCaml and C++. In all there were 81 teams using C++, 67 using C, 66 using Haskell, 64 using Python, 52 using OCaml, 48 with Java, and 35 with Perl. Only 4 tried Erlang (the same number as used Delphi).


According to the report from the competition:


...As far as functional programming is concerned, we must conclude that functional languages didn’t fare too well this year (although in the Top 15 there were five users of OCaml and three of Haskell). Better luck next year!


There were 421 teams, of which 122 used functional languages -- about 29% (which ought to be much higher than their rate of adoption by the industry). Still, over 50% of the top 15 places were won by the teams that used functional languages. It looks like good performance to me.


It says to me that we're still in the early days of widespread adoption of functional programming.


I don't think this is what it says. Functional programming languages are obviously less popular than imperative ones. Maybe functional languages will never be as widespread as imperative languages. However, this competition is just a competition. I doubt it says much about language adoption by the industry now or in the future. (By the way, has anyone heard of Ruby? :) ) If anything, this competition suggests that functional programming languages are quite widespread, given their high rate of adoption by the contestants.


Like Ruby, which had tiny uptake until Rails and 37 Signals boosted it into the stratosphere, functional programming languages lack a reason to use them.


If Ruby lacked a reason to be used before 37 Signals, why did DHH use it to create Rails? He could have stuck with PHP. The reason to use Ruby did exist, even if fewer people knew about it.

"Functional programming languages lack a reason to use them." I don't think that great concurrency, scalability, fault tolerance, distributed programming, hot code swapping, Mnesia, Yaws, and the rest are useless.

(In fact, Amazon has made Erlang much less useless today by announcing the new 4-core and 8-core options for EC2 instance. I can't wait to get my hands on those :) )


Even those who advocate them [functional languages] aren't comfortable enough with them to push on with them in the face of hard problems.


The teams that were comfortable with functional languages did push on them in the face of hard problems: those teams used functional languages in the contest. They took 8 of the top 15 places, too.

I must say that I disagree with the conclusions of the article.

I'll end this posting with a relevant message from the Erlang-questions mailing list yesterday:


> Hi.
> Recently, I have been trying to learn a language other than Java.
> Erlang seems to be ready for the prime time (used in the commercial world).
> On the other side, OCaml seems to be a cool and capable language.
> I want to learn something useful and fun.
> Tell what do you think ?
> (if you are Erlang die-hard, please put this aside ;))
>

When I was evaluating Python alternatives for building the core
technology behind MochiAds I tried out a bunch of languages and Erlang
was the only one that was easy for me to learn and had the right
balance of features, performance, and reliability. A year later we
have about 16 machines running 80 Erlang nodes powering about 16
different "components" of our infrastructure and 4 people working on
it at the moment (originally it was just me). It worked out so well
that we rewrote the server component of our MochiBot service in Erlang
and we've been using it to build lots of internal tools such as our
monitoring software, our single sign-on service, etc. as well. None of
us had previous Erlang experience, but we're all very comfortable with
it now.

After about a year with Erlang, I'm not sure I could part with hot
code loading, light-weight processes, and multiplexed socket IO for
writing servers. Also, Mnesia has been really useful to us to
temporarily store "real-time" data (ram_copies) so that we don't have
to make users wait for it to get batched into the SQL databases. The
distribution stuff mostly Just Works once you figure out how to set it
up (though we did have one bad experience with a network partition due
to a switch acting up, it was recoverable manually).

O'Caml is a useful language too, but for writing a network app I can't
really imagine going with anything but Erlang if you're looking for
redundancy and scale. Unless you want to write your own half-baked
Erlang-like system before even trying to solve something a little
closer to your actual problem domain.

-bob


It looks like functional programming must be good for something.

Monday, October 15, 2007

Blog Maintenance

I moved my blog to a different hosting provider. Please let me know if you notice any URLs that the move may have broken.

I also removed that annoying CAPTCHA plugin so you shouldn't have problems leaving comments from now on. If the plugin prevented you from leaving comments in the past you can now go ahead and leave them now. (I may have to put CAPTCHA back if Akismet flakes out on me but I hope I won't have to resort to that.)

I apologize for any inconvenience.

Sunday, October 14, 2007

More Vimagi Goodies

- You can now undo strokes.
- There's a better tools interface.
- You can embed paintings on any site, and even play them back!

For example, here's a great painting by Rivka. It's titled "horror flicks".



Enjoy!

Wednesday, October 10, 2007

If I Were Building Amazon.com...

I would use ErlyWeb :)

From Amazon's Dynamo by Werner Vogel:


"In Amazon̢۪s decentralized service oriented infrastructure, SLAs play an important role. For example a page request to one of the e-commerce sites typically requires the rendering engine to construct its response by sending requests to over 150 services. "


Unless all 150 service requests are executed sequentially, which is hard to believe, Amazon's page rendering ought to be able to benefit from Erlang's lightweight concurrency -- even more so than Vimagi :)

This makes me think about adding the following return value to ErlyWeb controller function:


{concurrent, [{ewc,...}, {ewc, ...}, {ewc, ...}]}


This would tell ErlyWeb to render each component in a different process. It may not make a big performance difference in rendering simple pages (it might even slow it down a bit), but for some applications (the future Erlmazon? :) ), it could be useful.

What do you think?

Thursday, October 04, 2007

Vimagi Updates

Vimagi news:

- Alpha channel support. You can now pick the transparency level from the color box.
- Clearing a painting erases all previous stroke data. From now on, the stuff you do before clearing the painting doesn't get played back.
- Performance improvements.
- Nightly backups to S3: your precious paintings are safe :)

Enjoy!

Wednesday, October 03, 2007

ErlyWeb is Facebook-Ready

In case you didn't know, Bryan Fink of BeerRiot fame has created a Facebook library for Erlang called erlang2facebook. You can get it from http://code.google.com/p/erlang2facebook/. It is bundled with a complete Hello World Facebook app that is built with ErlyWeb, so you can hit the ground running with your Erlang Facebook development.

I can think of some fun projects to build with this library. If you're going to use it to build an app, please let me know!

Thursday, September 27, 2007

Embedding Vimagi

Dear reader, I have good news. The internet is finished. The missing link was finally implemented.

You can now embed your Vimagi paintstream in any web page :)

All you have to do is add the following to your HTML, replacing [USERNAME] with your Vimagi username:


My Vimagi PaintStream






Enjoy!

Tuesday, September 25, 2007

Great Concurrency -> Better Webapps

I've been asked a few times: Is concurrent programming really useful for web developers? Web servers already handle client requests in different threads/processes, so application code doesn't need to use concurrency explicitly, right?

My opinion is that althought it's possible to write many database-driven web apps without using concurrency, concurrency can still be useful.

Vimagi has some real-time features in that rely heavily on concurrency, but it also helps in a more common scenario: speeding up page loads.

In Vimagi, every time someone views a painting or a user profile, a SQL query is executed to increment the view counter for that page. This is more-or-less what the controller code does:


show(A, Id) ->
painting:increment([num_views], {id,'=',Id}),
Painting = find_id(Id),
{data, Painting}.


This caused a problem. When traffic increased, the 'increment' function, which executes an UPDATE statement in the database, started performing poorly, thereby noticeably slowing down page loads. Occasionally, it even timed out after 5 seconds, showing an error screen to the user.

I increased the timeout to 20 seconds. This helped avoid the error screen, but the response time was still too long.

Erlang made this almost too easy to fix. All I had to do was change


painting:increment([num_views], {id,'=',Id})


to


spawn(
fun()->
painting:increment([num_views], {id,'=',Id})
end)


It's darn simple, and it works. Instead of executing the UPDATE statement in the process that's responsible for rendering the page, I spawn a new process and execute the transaction there. Spawning processes in Erlang is dirt-cheap, so I don't have to worry about performance or memory issues. The user gets a quick response, the view counter is incremented, and the problem goes away.

Sunday, September 23, 2007

Introducing Vimagi.com

I just launched Vimagi, my first ErlyWeb app: vimagi.com.

Painting is fun. Sharing paintings with your friends is also fun. Painting together with your friends is a new kind of fun you can have on Vimagi.

Give it a try and let me know what you think -- I'll appreciate the feedback.

Note: Vimagi is in early beta. It probably has many bugs. Please email me if you find any.

Saturday, September 22, 2007

Fancy Language/Framework Poll Results

My Wufoo poll has reached 99 entries! Let's check out the results.

Most popular languages:
Ruby: 31%
Erlang: 23%
Python: 21%
Lisp: 4%

Most popular frameworks:
Ruby on Rails: 39%
All web frameworks suck: 16%
Django: 15%
ErlyWeb: 6%

You can see the report here, with pretty graphs: http://yariv.wufoo.com/reports/a-fancy-languageframework-report/.

(Btw, it would be great if Wufoo allowed embedding public reports on other pages.)

Ruby/Ruby on Rails is the most popular language/framework among my blog readers. This is interesting, given that I don't write about Ruby much. My take on it is that many Rubyists read my blog because a) they are numerous (I'm sure there are many more Ruby hackers than Erlang hackers) and b) they are curious about other languages, especially Erlang. The same goes for Pythonistas.

It's interesting how many people think that most web frameworks suck. Maybe they don't do web programming (but then, why would they think tools they never use suck?), they love the PHP/JSP model (somehow, I doubt it), or they have better solutions for their needs. It's a mystery.

A couple of responders hate all computer languages. I wonder why they are reading my blog, then. The self-help section is yet to be launched... :)

A curious response is by a Lisp hacker who likes ASP.NET.

Only a couple of responders like Java. It's strange, given that Java is one of the most widespread languages. Maybe I can make ErlyWeb more marketable by renaming it 'Erlang Enterprise ConcurrentBean Application Server' :)

This is too much fun.

Friday, September 07, 2007

A Fancy Language/Web Framework Poll

Making forms with wufoo is so darn easy I just had to create one. It is below, in all its glory: a language/web framework poll for my blog readers.


Powered by Wufoo

Thanks for participating. Your input is very valuable -- I think.

Saturday, August 11, 2007

SF Bay Area ErlLounge, Aug 15

In case you haven't seen the announcement on the mailing list (http://www.erlang.org/pipermail/erlang-questions/2007-August/028324.html), the 2nd SF Bay Area ErlLounge will take place on Aug 15. Anyone who's interested is welcome to attend.

Saturday, August 04, 2007

BeerRiot - The First ErlyWeb App

Many people may not know this, but ErlyWeb has an app.

It's called BeerRiot (http://beerriot.com), and its creator is Bryan Fink, whose been brewing it for a few months now (pun intended). He blogs about it at http://blog.beerriot.com/. (My favorite quote: "To those who are considering Erlang and ErlyWeb as a framework for their website: just go for it.")

BeerRiot is the quintessential web destination for beer aficionados. It has a beer database, user-comments, votes, and tags. It even comes in black and white background flavors. The only feature it lacks is letting you taste the beers somehow -- I guess I'll have to add this capability to the ErlyWeb todo list :)

According to my highly unscientific and subjective performance tests, BeerRiot feels very fast. I measured page load time with YSlow, which reported just 0.2 seconds for the homepage.

Kudos to Bryan for making such a nice app, and for being the first person to create a production ErlyWeb app.

Sunday, July 22, 2007

Erlang-Style Concurrency in Google Gears

I heard about Google Gears when it came out a few weeks ago, but only a couple of days I took it for a short test drive. I was surprised to see that Google Gears uses Erlang-style concurrency (well, sort of).

Google Gears supports asynchronous background processing using the WorkerPool module, which lets your Google Gears app execute multiple tasks concurrently without blocking the main UI thread. What I find interesting thing about WorkerPool is that threads only communicate by message passing and that there is no API for synchronizing access to objects that are passed around between threads. This is a departure from the traditional concurrency toolbox in imperative languages -- using synchronized access to shared memory.

It's so easy to use it's almost... Erlangy :)

So how does Google Gears pulls off this feat of simplicity?

Google Gears only lets you send strings between threads. Since Javascript strings are immutable, there's no danger in letting multiple threads share them.

This solution is clever, but it comes at a cost -- you must serialize your objects before sending them between threads, which incurs a performance penalty (some extra coding, too). I don't expect the performance penalty to noticeably impact most Google Gears apps, though, which are mostly offline adaptations of standard webapps.

Google Gears has managed to pull off easy concurrency in an imperative language by relying on the immutability of strings in Javascript, but this approach is only practical on the desktop. In a server app with higher performance and scalability requirements, sending serialized objects would probably slow things down too much.

All in all, I think Google Gears is a great solution for adding offline support to webapps, and it gets bonus points for using Erlang-style concurrency. Now I'm just waiting for Safari support :)

Wednesday, July 11, 2007

San Francisco ErlLounge, Jul 18th

About a month ago, I packed my stuff, said farewell to Boston, and moved to the San Francisco Bay Area. I currently live in the town of San Mateo, where occasionally cars drive by and people walk their dogs. It's very exciting. I will soon move to Palo Alto for a few months, and then who knows. Whatever it is, it won't be as lively as San Mateo.

I'm planning on attending the San Francisco ErlLounge on Jul 18th. Visit http://umichigan.facebook.com/p.php?i=2223888&k=33553ba7bd for details. You should attend if you can. I'll tell you thrilling stories about San Mateo. I promise.

On an unrelated note, the spammers got me beat, and I finally gave up and disabled comments on this blog. If you want to share your thoughts, you can email me at yarivsblog at gmail dot com.

Friday, June 08, 2007

Concurrency Buzz

There's been quite a buzz lately in the programming blogsphere about concurrent programming. I think the software world is paying increasing attention to concurrent programming (and Erlang) due to the following reasons:

- The apparent decline of Moor's law coupled with the rise of multi-core CPUs.
- The "traditional" (i.e. shared-memory based concurrency) employed by most languages is flawed: it's too hard to grasp, it's too error-prone, and it's not multi-core friendly.
- Joe Armstrong's book, Programming Erlang has made Erlang accessible to a wide audience.
- Reddit.

There's more to Erlang style concurrency than lightweight processes and message passing. I don't expect other (popular) languages to fully gain Erlang-style concurrency (not soon, at least) because to do so they would have to adopt the following combination of traits:

- A scheduler that uses a mixture of lightweight processes and system threads (SMP Erlang spawns a system thread per CPU).
- Immutable variables.
- General avoidance of long blocking calls to native code in libraries.
- A garbage collector that utilizes per-process heaps to avoid freezing the entire VM during sweeps.
- A revised approach to error handling based on supervision trees and non-defensive coding.
- Web servers, database drivers, libraries, tools, etc. that actually use the new concurrency primitives.

As the saying goes, never say "never"... :)

Does that mean that you need Erlang to make your applications utilize available CPUs on multi-core machines? Of course not. However, when 80 core machines start appearing in our data centers in a few years, at least some people will take comfort in Joe Armstrong's statement:

"Your Erlang program should just run N times faster on an N core processor"

Thursday, April 19, 2007

SlideAware: From Python to Ruby to Erlang

SlideAware has created a slick web-based solution for managing the life cycle of Powerpoint presentations. They started using Python, then they switched to Rails, and finally settled on pure Erlang. Using Erlang, they replaced a combination of Lighttpd + RoR + SQLlite + XMLRpc + Jython + Lucene with the much simpler and highly scalable stack of Yaws + Mnesia + Erlang.

They wrote an interesting article describing their technology choices. Check it out at http://slideaware.typepad.com/slideaware/2007/04/from_python_to_.html.

Tuesday, April 17, 2007

Announcing ErlyWeb 0.6

I'm pleased to announce the release of ErlyWeb 0.6. This very exciting release contains a new Mnesia driver for ErlyDB created by Matthew Pflueger and a Postgres driver (based on the open source driver from Erlang Consulting) created by Roberto Saccon. Finally, you can use a single database abstraction layer in Erlang to access MySQL, Postrgres and Mnesia without writing a single line of SQL or DBMS-specific logic :)

You can get ErlyWeb 0.6 from the ErlyWeb website or from its Google Code repository. Make sure to check out CHANGELOG.txt for a complete list of changes.

If you have any questions regarding the new drivers, please ask the ErlyWeb Google Group.

An Erlang Pipe

Yahoo Pipes is one of the most innovative apps I've seen: it lets you program data feeds. More specifically, you can use Pipes build programs (called "pipes") that generate output feeds from input feeds by combining, filtering, and manipulating the inputs using different modules from the Pipes IDE. The IDE has a friendly graphical interface, letting you drag, drop and connect different components easily. The IDE is implemented in Javascript, which is impressive.

I couldn't resist playing with Pipes, so I created an Erlang pipe. The Erlang pipe reads the feeds from programming.reddit.com and lambda-the-ultimate.org, combines them, and filters the results by accepting only articles whose contents contain the word 'Erlang'. It then combines the result with the inputs from planeterlang.org.

Check out this pipe at http://pipes.yahoo.com/pipes/pipe.info?_id=4jVpGgjt2xGttygedrq02Q.

Sunday, April 15, 2007

Thursday, March 15, 2007

Boston BarCamp2

Is anybody who reads this blog going?

http://barcamp.org/BarCampBoston2

I think I may go and try to talk some people into using Erlang in their startups.

Update (2 hours later):

It looks like my job is going to be easier than expected. I checked out the at the BarCamp real time visual aggregator (http://barcamp.org/BarCampNewsVisualAggregator), and guess what I saw?


TECH. PROBLEMS

This experiment may cause to much traffic for pbwiki... to be improved...


Maybe yaws could be a better platform for serving the BarCamp aggregator: http://www.sics.se/~joe/apachevsyaws.html :)

Thursday, March 01, 2007

Announcing ErlyWeb 0.5

I'm pleased to announce that ErlyWeb 0.5, the biggest and best release yet, is now available to download from erlyweb.org. There's a long list of improvements and fixes, of which you can read here.

As I indicated in a previous posting, ErlyWeb no longer supports app views. If you are upgrading from an earlier version, you should convert your app view to an html container and use the 'phased' return value from the app controller, as described in the updated documentation on the website.

ErlyWeb is still in alpha, and will likely remain this way until somebody has used it to build a complete production app. So, who will reach the finish line first? :)

Friday, February 23, 2007

ErlyWeb Tutorial: Life In the Intersection of FP and Dynamic HTML

Web applications often have pages in which the user can edit the fields of an existing database record and then save the updated record by submitting the form. To generate such an 'edit' page, you generally have to read the record from the database, generate an HTML form whose fields are pre-populated with the record's data, and then send the form back to the browser. Let's look at a few ways of doing this with ErlyWeb (for instructional purposes, we won't be using "-erlyweb_magic(on)." :) ).

Let's suppose we want to provide a form for editing the records of a database table called 'language'. We can create an ErlyDB module for working with this table as follows:

language.erl:


-module(language).


When we call erlyweb:compile() (or when ErlyWeb auto compiles our modules becase we have turned 'auto_compile' on, as one should do during development), ErlyDB reads the metadata for the 'language' database table and generates a large number of functions in the 'language' module covering the whole exciting CRUD gamut.

Now let's create a basic controller implementing the data access logic for the 'language edit' page. I will only show the most stripped-down logic necessary for rendering the 'edit' form, which is the focus of this tutorial.

language_controller.erl:

-module(language_controller).
-export([edit/2]).

edit(A, Id) ->
case language:find_id(Id) of
undefined ->
{data, {error, no_such_record}};
Language ->
{data, Language}
end.


When ErlyWeb receives a request such as "http://myapp.com/language/edit/1", it will invoke the 'edit/2' function in language_controller. This function will look up the database record using a SQL query such as "SELECT * FROM language WHERE id=1", and send the resulting record directly to the view function, or the tuple {error, no_such_record} if no record matches the requested id.

Now let's take a first pass at making a the view for this component (note that in this tutorial, we'll avoid any fancy automatic form-generation methods and write most of the HTML by hand).

language_view.et:

<%@ edit({error, no_such_record}) %>
ouch, the record doesn't exist


<%@ edit(Language) %>

name:


paradigm:

....




Looks good so far? Well, it's not really supposed to, because this code has some problems.

The most urgent problem is that this code doesn't check that the record's fields are formatted as iolists. If any of the record's values is 'undefined', for example, this code will cause the process to crash (by 'crash' I don't mean the catastrophic definition of 'crash' that is used with most programming languages, but the Erlang definition, which means the lightweight process in which the crash occurs has died -- an isolated event inside the VM, nothing more).

We can address this issue in a few different ways. I'll show you one simple way, in which we create a function in the view module to do the proper conversion. Using this method, this would be the modified code for the new language.et:


<%@ edit(Language) %>

name:


paradigm:

....



<%@ show(Language, Field) %>
<% erlydb_base:field_to_iolist(language:Field(Language)) %>


Note that this code uses the Erlang way of doing dynamic function dispatch: you use a variable in place of the module's function (and/or the module itself) you want to call. This is an extremely useful feature -- know it and it will serve you well.

The code works, but it irks me somewhat. The issue is that I don't want the view logic to know anything about ErlyDB records and how to access and format their fields. Views are supposed to be simple, and should only use the most rudimentary logic required to render their parameters. Putting ErlyDB logic in the view violates this rule.

So, how do we fix this issue? We have a few options. One option is for the controller to pass the view a list of pre-rendered fields instead of the Language record itself. This could be done in the controller by returning the following:


{data, [erlydb_base:field_to_iolist(language:Field(Language)) ||
Field <- language:db_field_names()]}


To handle this list, the view function would have to change, too:


<%@ edit([Name, Paradigm, ....]) %>

name:


paradigm:

....


Using this approach, we've succeeded at stripping the view function of ErlyDB-related logic, but there is a downside: we now have to maintain as the function parameter a list of field variables that matches exactly the length and order of the field list returned from language:db_field_names(). This is acceptable for records whose field list is small and doesn't change much, but for records with large numbers of fields this quickly becomes a pain to maintain.

Our search for the ideal solution isn't done, but from my experience with Erlang in specific and functional programming in general, the ideal solution -- elegant, clean, flexible -- almost always exists, even if you haven't found it yet :)

What we ultimately want is to be able to look up properly formatted field values by name without worrying about the fields' order and without calling ErlyDB functions (or functions generated by ErlyDB) directly. To do this, we can go the "traditional" (i.e. lame :) ) route of passing the view function a data structure mapping field name keys to data values. In Erlang, this could be a simple property list, a dict (i.e. a hash table), a gb_trees structure, or some other associative container. With this approach, the view function would populate the field values using snippets such as


<% dict:find(name, LanguageDict) %>


However, I don't like this approach because it incurs the overhead of creating and using the associative container (in a real app, this overhead would probably be negligible, but I still don't like it :) ), and it's tedious to repeatedly call the associative container functions to look up field values. In addition, real purists would argue that the view function shouldn't know the type of the container, or even that we're using some associative container to look up the values of our fields in the first place!

FP to the rescue.

My favorite solution to our view dilemma is to change the controller's 'edit' function as follows:


edit(A, Id) ->
case language:find_id(Id) of
undefined ->
{data, {error, no_such_record}};
Language ->
{data,
fun(Field) ->
erlydb_base:field_to_iolist(language:Field(Language))
end}
end.


What's going on here? Instead of passing the view function some data structure, we pass it a closure, i.e. a function whose logic uses one or more variables that exist in the closure's lexical scope. Using this closure, we can get the properly formatted value for any field of the Language record with minimal performance -- and coding -- overhead. Here's how we use it in the view function:


<%@ edit(F) %>

name:


paradigm:

....


As you can see, using closures in this scenario gives us the best of all worlds: almost no performance overhead, minimal code, no maintenance of field lists, and clean separation between controller and view logic.

In summary, if you want to keep your code clean, simple, and properly decoupled, closures are often your best friends.

Monday, February 12, 2007

Introducing... The ErlyWeb Logo!

My plea for help in the graphics design department in the making of an ErlyWeb logo has yielded a few candidates, and although I sincerely appreciate the efforts of the people who made them, and they gave me some good ideas, none of them has truly captured my fancy. So, I decided to take matters into my own hands and finally create a real logo for ErlyWeb.

Without further ado, I hereby present ErlyWeb's shiny new logo:





(This is the big version. A small version is on http://erlyweb.org.)

This logo is inspired by Erlang, Big Sur and Lisa Simpson.

(When I wrote that my graphic design skills were non-existent, you didn't think it really meant there was a Picasso inside me trying to break out, did you? :-) )

Wednesday, February 07, 2007

ErlyWeb Tutorial: Using Components To Create Dynamic Containers

ErlyWeb's app views are a basic mechanism for sharing common view elements between components. An app view is a simple Erlang module or ErlTL template file that nests pre-rendered components in static (generally HTML) content. Most simple applications probably have an app view that looks like this:




My App


<% Data %>




This mechanism works well when your components share only static view elements. But what do you do when your components share dynamic view elements? If this is the case, you have a few options:

- You could put the logic for generating the dynamic content in the view module. However, this would be ugly, and it would go against one of ErlyWeb's primary purposes, which is to help you create a clean separation between application logic and view logic. Let's forget this option and move on.

- If all your components share an identical set of dynamic elements, you can change your app controller's hook function, so instead of returning '{ewc, A}', it returns a list containing the 'ewc' and/or 'data' tuples common to all components. For example, if all your pages have sidebar, your hook/1 function could return '[{ewc, sidebar, [A]}, {ewc, A}]'. This tells ErlyWeb render each component in the list, and then pass into the app view a 2 element list, wherein the first element is the rendered sidebar and the second element is the rendered component that corresponds to the browser's request. To properly handle this new parameter type, you could change your app view as follows:


<%? [Sidebar, Content] = Data %>


My App







<% Sidebar %> <% Content %>





Again, this works well when all your pages have the same sidebar component. However, what if different pages have different sidebar components? Or what if some pages have a sidebar and some don't? Let's look at the next option.

- Starting from ErlyWeb 0.4, your controllers can return a {response, Elems} tuple, whose Elems list may include the {app_view_param, Ewc} tuple (for more info on the 'response' tuple, check out the ErlyWeb docs. Similar to the first option, this response tells ErlyWeb to render the (list of) 'ewc' and/or 'data' tuples contained in the Ewc variable, and then pass into the app view a 2-element list containing the rendered components followed by the rendered content for the request. In addition, you can use the {app_view, ModuleName} tuple to pick a specific app view (or disable the app view entirely) instead of using the default one.

This approach works, and it offers greater conveniece in customizing the dynamic app view data for each component (or each component function), but I don't like it. The reason is that I don't think that components should know or control what happens in the areas of the page outside of their own rendered HTML. In addition, it gets confusing if multiple nested sub-components define the 'app_view_param' element in their responses: which one should ErlyWeb be use? Given these shortcomings, I'm considering deprecating if not outright removing support for the 'app_view' and 'app_view_param' tuples from the 'response' tuple in ErlyWeb (it is and an experimental feature, after all, as the documentation notes). So, let's look at the next option.

- The last, and arguably best approach, is to use ErlyWeb's component system to create dynamic containers. A container is a component that contains one or more sub-components, which are given to the container as parameters. The container tells ErlyWeb to render its parameters together with any dynamic data specific to the container. For example, to create a container that renders its sub-component(s) together with a sidebar, you could use the following code:

sidebar_container_controller.erl:

-module(sidebar_container_controller).
-compile(export_all).

%% This tells ErlyWeb to reject browser requests that try to
%% access this component directly
private() -> true.

index(A, Ewc) ->
[{ewc, sidebar, A}, Ewc].


sidebar_container_view.et:

<% Data %>

<%@ index([Sidebar, Contents]) %>





<% Sidebar %><% Contents %>



Using this container, any component can render a subcomponent with a sidebar by simply returning the tuple '{ewc, sidebar_container, [A, {ewc, subcomponent, [A]}]}'. What I like about this container, beside its flexibility and reusability, is that it doesn't require any special logic nor does it rely on any features beyond ErlyWeb's simple and elegant component model.

An ideal place to use this container is in the app controller's hook function. For example, to achieve the same effect as the app view-based approach from the second option, you could implement this function as follows:


hook(A) ->
Ewc = erlyweb:get_initial_ewc({ewc, A}, myapp_erlyweb_data),
{ewc, sidebar_container, [A, Ewc]}.


(The first line may look a bit cryptic -- indeed, it relies on a function that isn't included yet in the official distribution -- but just know that it enforces the 'private' policies on component requests that come from clients.)

A nice thing about this approach is that it helps simplify the app view, which doesn't require unpacking the Data parameter anymore and laying out the sidebar and the contents as the second example illustrates.

We could take this a step further and make the app view obsolete by creating an HTML page container as follows:

html_container_controller.erl:

-module(html_container_controller).
-compile(export_all).

private() -> true.

index(A, Ewc) ->
Ewc.


html_container_view.et:

<% Data %>

<%@ index(Data) %>


My App


<% Data %>




Now, we could implement hook/1 as follows:


hook(A) ->
Ewc = erlyweb:get_initial_ewc({ewc, A}, myapp_erlyweb_data),
{ewc, html_container,
[A, {ewc, sidebar_container, [A, Ewc]}]}.


With this container structure, out app view becomes


<% Data %>


As you can see, using the container approach, we've replaced the somewhat crippled app view with a much more flexible and powerful ErlyWeb component. Not bad, huh?

So where does this lead us? Will app views soon be a thing of the past? I think this may very well happen :)

Thursday, January 11, 2007

ErlyWeb Tutorial: Creating a Simple Login Page

Let's see how to create a simple login component with ErlyWeb.

Update: I simplified the HTML to reduce clutter.

First, let's take a look at login_view.et:


<% Data %>

<%@ index({Username, Errs}) %>
<% [error(Err) || Err <- Errs] %>


login

username:

password:




<%@ error(Err) %>
- <% error_msg(Err) %>


<%@ error_msg({value_too_short, FieldName, Min}) %>The <% FieldName %> must be at least <% Min %> characters.
<%@ error_msg(invalid_credentials) %>Invalid username/password.


As you can see, the 'index' function of login_view.et accepts a 2 element tuple. The first element is the value of the 'username' field, and the second element contains a list of error values. An error value can be any Erlang data type. The error_msg/1 function maps error values to their textual message. This is done by simple pattern-matching (aside: damn, I love pattern-matching :) ).

Before we take a look at the controller, let's implement a nice little helper function that contains some boiler-plate code for form validation.

Update: The following is generic, reusable code that can work for any form validation, not just login pages. It may even be included in the framework at some point.


validate(A, Fields, Fun) ->
Params = yaws_api:parse_post(A),
lists:foldl(
fun(Field, {Vals, Errs}) ->
case proplists:lookup(Field, Params) of
none -> exit({missing_param, Field});
{_, Val} ->
Val1 = case Val of undefined -> ""; _ -> Val end,
Acc1 =
case Fun(Field, Val1) of
ok ->
{[Val1 | Vals], Errs};
Err ->
{[Val1 | Vals], [Err | Errs]}
end,
Acc1
end
end, {[], []}, lists:reverse(Fields)).


This function takes the Yaws arg record, a list of expected field names to validate, and a function object that takes 2 parameters -- the field name and the value -- and returns 'ok' or an error code. The validate/3 function returns a 2 element tuple. The first element is the list of values corresponding to the field names, and the second element is a list of error codes returned by the function object as it was applied to each field. validate/3 also converts empty field values from 'undefined' to empty strings so they can be passed to the view component, which always returns iolists, verbatim.

Now, let's take a look at login_controller.erl:


index(A) ->
case yaws_arg:method(A) of
'GET' ->
{data, {"", []}};
'POST' ->
case validate(A, ["username", "password"], fun validate_field/2) of
{[Username, Password], []} ->
case app_user:find(
{'and',
[{username,'=',Username},
{password,'=',crypto:sha(UserName ++ Password)}]}) of
[] ->
{data, {Username, [invalid_credentials]}};
[User] ->
Key = crypto:rand_bytes(20),
Encoded = http_base_64:encode(binary_to_list(Key)),
app_user:save(app_user:key(User, Key)),
{response,
[yaws_api:setcookie("key", Encoded),
{ewr, main}]}
end;
{[Username, _Password], Errors} ->
{data, {Username, Errors}}
end
end.

validate_field("username", Val) when length(Val) > 4 -> ok;
validate_field("username", _) -> {value_too_short, "username", "5"};
validate_field("password", Val) when length(Val) > 5 -> ok;
validate_field("password", _) -> {value_too_short, "password", "6"} .


If the request is a GET, login_controller passes into the view function an empty string for the username and no errors. The it's a POST, login_controller checks that the fields have passed basic validation, i.e. their values pass the minimum length requirements. If so, it checks that the username/password hash combination is in the database. If is it, login_controller assigns the user a random 20 byte key string as a cookie value, which it also saves in the database (in this example, we use the database for session management, but this isn't required) and then redirects to the 'main' component (note: this 'response' tuple is only available in ErlyWeb 0.4, which hasn't been released yet). If the username/password combination is invalid, login_controller passes to the view function the previous Username value and the 'invalid_credentials' error code.

This example illustrates a few cool things about Erlang/ErlyWeb:

- ErlyDB makes working with the underying RDBMS very simple and natural.
- Pattern matching is a joy, both in validation code and in ErlTL templates.
- ErlSQL lets us create short SQL statement snippets dynamically while avoiding manual iolist construction and guaranteeing protection against SQL injection attacks.
- Everything is so easy! You gotta love Erlang :)

Alright, now back to work.

Tuesday, January 02, 2007

My Talk at MSLUG

On January 17th, I'll be giving a talk about my adventures in open source Erlang at the Montreal Scheme/Lisp User Group (MSLUG) meeting. For more information, visit http://schemeway.dyndns.org/mslug.

I hear the members of this group have a strong interest in Erlang. Guillaume Germain, who wrote Termite, an implementation of Erlang style concurrency for Gambit-C Scheme, is a member, and so is Marc Feeley, the author of Gamit-C. I'm looking forward for my second opportunity to be surrounded by people who are interested in Erlang :)

This meeting is open to everyone, so if you're interested in attending, you're more than welcome.

Aside: I lived in Montreal for a year when I was 15. My last visit was about 5 years ago, so It'll be nice to visit again. It's been a while since I've had some good poutine! :)

Saturday, December 30, 2006

New Year's Brain Teaser

Here's a good brain teaser to give you some entertainment for the new year.

You have 12 balls, identical in every way except that one of them has a different weight from all the others. You also have a balance scale. In 3 rounds, in each of which you can compare the weights of any 2 groups of balls, you have to ascertain which is the ball with the different weight and whether this ball is heavier or lighter than the other balls.

This is not a trick question in any way. I solved part of this puzzle after I got a significant hint. I wish I had tried a bit harder to solve it by myself, but it's too late now. If you want a hint, you can email me, but you'll feel greater satisfaction if you solve it by yourself.

Happy new year!

Tuesday, December 19, 2006

ErlyWeb is Undocumented No More!

I resisted the distractions. Even the killer duo of Lord of the Rings books and the World Wide Web couldn't stop me (although it did slow me down considerably :) ). I finally managed to sit down on my ass until I finished writing the first draft of the ErlyWeb API documentation, in meticulous conformance to the edoc specification!

Yes, my friends, it's true. You can now go to http://erlyweb.org/ and feast your eyes on the most glorious API documentation ever written! Well, ok, I admit that may be a slight exaggeration, but it sure beats having to rely on a bunch of tutorials and source code, doesn't it? :)

If you find any glaring holes or errors, please let me know.

(To generate the HTML yourself, check out the latest version from trunk and do the equivalent of "edoc:application('ErlyWeb', "/path/to/erlyweb/trunk", [no_packages])", and the files will appear in erlyweb/doc.)

Enjoy!