Friday, October 27, 2006

Introducing ErlyWeb: The Erlang Twist on Web Frameworks

If you've been reading this blog for a while and you've been connecting the dots, you probably know where this is going... :) I you haven't, that's ok -- my old postings aren't going anywhere. There's a lot of Erlang goodness in them for you to soak up at your leisure as you're getting up to speed on this great language.

Without further ado, I present to you the culmination of all of my exciting adventures thus far in the land of open source Erlang:

ErlyWeb: The Erlang Twist on Web Frameworks.

Don't worry, I'm not going to blab for a long time now about why I think ErlyWeb + Yaws is the best web development toolkit available (not that I'm biased or anything :) ). Instead, I decided I'll just take you on a quick tour of how to use ErlyWeb, and let you use your own knowledge about Erlang to fill in the gaps :)

- Get the latest ErlyWeb archive from erlyweb.org, unzip it, and put the zip file's contents in your Erlang code path. (The Erlang code path is the root directory within which the Erlang VM searches for compiled modules. In OS X, it's "/usr/local/lib/erlang/lib". For more information, visit http://www.erlang.org/doc/doc-5.5.4/lib/kernel-2.11.4/doc/html/code.html).

- Download and install Yaws if you don't already have it.

- Start Yaws in interactive mode ("yaws -i") and type in the Yaws shell


erlyweb:create_app("music", "/apps").


(I'm assuming that "/apps" is the parent directory of your Yaws apps.)

This will create an ErlyWeb directory structure as well as a few files. (Note: this initial procedure will probably be shorter when ErlyWeb matures.) This is what you should see:


/apps/music
/apps/music/ebin
/apps/music/src/music_app_controller.erl
/apps/music/src/music_app_view.et
/apps/music/src/components
/apps/music/www
/apps/music/www/index.html
/apps/music/www/style.css


- Edit your yaws.conf file by adding a server configuration with the following docroot, appmod, and opaque directives, then type "yaws:restart()."


docroot = /apps/music/www
appmods = <"/music", erlyweb>

appname = music



- Open your browser and point it at http://localhost:8000/music (note: your host/port may be different, depending on your Yaws configuration). You should see the following page, (breathtaking in its design and overflowing with aesthetic genius, if I may add):

erlyweb_index.png

- Create a MySQL database called 'music' with the following code (thanks, Wikipedia :) ):


CREATE TABLE musician (
id integer primary key auto_increment,
name varchar(20),
birth_date date,
instrument enum("guitar", "piano",
"drums", "vocals"),
bio text
) type=INNODB;

INSERT INTO musician(name, birth_date,
instrument, bio) VALUES
("John Lennon", "1940/10/9", "vocals",
"An iconic English 20th century
rock and roll songwriter and singer..."),
("Paul McCartney", "1942/6/18", "piano",
"Sir James Paul McCartney
is a popular Grammy Award-winning
English artist..."),
("George Harrison", "1943/2/24", "guitar",
"George Harrison was a popular English
musician best known as a member of The Beatles..."),
("Ringo Star", "1940/7/7", "drums",
"Richard Starkey, known by his stage name
Ringo Starr, is an English popular musician,
singer, and actor, best known as the
drummer for The Beatles...");



- Back in Yaws, type


erlyweb:create_component("musician", "/apps/music").


This will create the following files:

/apps/music/components/musician.erl

-module(musician).


/apps/music/components/musician_controller.erl

-module(musician_controller).
-erlyweb_magic(on).


/apps/music/components/musician_view.erl

-module(musician_view).
-erlyweb_magic(on).


Back in Yaws, type


erlydb:start(mysql, [{hostname, "localhost"}, {username, "username"},
{password, "password"}, {database, "music"}]).
erlyweb:compile("/apps/music", [{erlydb_driver, mysql}]).


(The erlydb_driver option tells ErlyWeb which database driver to use for generating ErlyDB code for the models. Note: this may change in a future version.)

Now go to http://localhost:8000/music/musician, click around, and you'll see the following screens:

erlyweb_list.png

erlyweb_new.png

erlyweb_edit.png

erlyweb_delete.png

"Aha!" you may be thinking now, "I bet he's using some Smerl trickery to call functions that contain mountains of horrible code only comprehensible to Swedish Field Medal winners!"

Well.. um, not exactly. In fact, this is the code for erlyweb_controller.erl


%% @title erlyweb_controller
%% @author Yariv Sadan (yarivsblog@gmail.com, http://yarivsblog.com)
%%
%% @doc This file contains basic CRUD controller logic. It's intended
%% for demonstration purposes, but not for production use.
%%
%% @license For license information see LICENSE.txt

-module(erlyweb_controller).
-author("Yariv Sadan (yarivsblog@gmail.com, http://yarivsblog.com)").

-export([
index/2,
list/2,
list/3,
new/2,
edit/3,
delete/3
]).

-define(RECORDS_PER_PAGE, 10).

index(_A, Model) ->
{ewr, Model, list, [1]}.

list(A, Model) ->
list(A, Model, 1).

list(A, Model, Page) when is_list(Page) ->
list(A, Model, list_to_integer(Page));

list(A, Model, Page) when is_integer(Page) ->
Records = Model:find_range((Page - 1) * ?RECORDS_PER_PAGE,
?RECORDS_PER_PAGE),

%% this function makes the 'edit' links in the record ids
ToIoListFun =
fun(Val, Field) ->
case erlydb_field:name(Field) of
id ->
Id = Model:field_to_iolist(Val),
erlyweb_html:a(
[erlyweb_util:get_app_root(A),
atom_to_list(Model),
<<"edit">>, Id], Id);
_ ->
default
end
end,
{data, {erlyweb_util:get_appname(A),
atom_to_list(Model),
Model:db_field_names_bin(),
Model:to_iolist(Records, ToIoListFun)}}.

new(A, Model) ->
Rec = Model:new(),
new_or_edit(A, Model, Rec).

edit(A, Model, Id) ->
Rec = Model:find_id(Id),
new_or_edit(A, Model, Rec).

new_or_edit(A, Model, Record) ->
Fields = tl(Model:db_fields()),
Vals = tl(Model:to_iolist(Record)),
Combined = lists:zip(Fields, Vals),
IdStr = case Model:id(Record) of
undefined -> [];
Id -> integer_to_list(Id)
end,
case yaws_arg:method(A) of
'GET' ->
FieldData = [{erlydb_field:name_bin(Field),
erlydb_field:html_input_type(Field),
erlydb_field:modifier(Field),
Val} || {Field, Val} <- Combined],
{data, {erlyweb_util:get_app_root(A),
atom_to_list(Model),
IdStr,
yaws_arg:server_path(A),
FieldData}};
'POST' ->
NewVals = yaws_api:parse_post(A),
Record1 = Model:set_fields_from_strs(Record, NewVals),
Model:save(Record1),
{ewr, Model, list}
end.

delete(A, Model, Id) ->
case yaws_arg:method(A) of
'GET' ->
Record = Model:find_id(Id),
Fields = [erlydb_field:name_bin(Field) ||
Field <- Model:db_fields()],
Vals = Model:to_iolist(Record),
Combined =
lists:zipwith(
fun(Field, Val) -> [Field, Val] end,
Fields, Vals),

{data, {erlyweb_util:get_app_root(A),
atom_to_list(Model), Id,
Combined}};
'POST' ->
Model:delete_id(Id),
{ewr, Model, list}
end.


And this is the code for erlyweb_view.et

<%~
%% @title erlyweb_view.et
%% @doc This is a generic view template for making simple CRUD
%% pages with ErlyWeb. It's intended for demonstration purposes,
%% but not for production use.
%%
%% @license for license information see LICENSE.txt

-author("Yariv Sadan (yarivsblog@gmail.com, http://yarivsblog.com)").
-import(erlyweb_html, [a/2, table/1, table/2, form/3]).

%% You can add component-specific headers and footers around the Data
%% element below.
%>
<% Data %>

<%@ list({AppRoot, Model, Fields, Records}) %>
<% a(["", AppRoot, Model, <<"new">>], <<"create new">>) %>

Records of '<% Model %>'
<% table(Records, Fields) %>

<%@ new({_AppRoot, Model, _Id, Action, FieldData}) %>
Create a new <% Model %>:
<% form(Action, <<"new">>, FieldData) %>

<%@ edit({AppRoot, Model, Id, Action, FieldData}) %>
delete


<% form(Action, <<"edit">>, FieldData) %>

<%@ delete({AppRoot, Model, Id, Combined}) %>
Are you sure you want to delete this <% Model %>?
<% table(Combined) %>
method="post">

onclick="location.href='<% AppRoot %>/<% Model%>'"
value="no">



Not exactly the stuff that would win anyone the Field Medal, if I dare say so.

If ErlyDB hasn't convinced you that Erlang is a very flexible language, I hope that ErlyWeb does. In fact, I don't know of any other language that has Erlang's combination of flexibility, elegance and power. (If such a language existed, I wouldn't be using Erlang :) ).

The flexibility of components

The notion of component reusability is central to ErlyWeb's design. In ErlyWeb, each component is made of a view and a controller, whose files are placed in 'src/components'. All controller functions must accept as their first parameter the Yaws Arg for the HTTP request, and they may return any value that Yaws accepts (yes, even ehtml, but ehtml can't be nested in other components). In addition, they can return a few special values:


{data, Data}
{ewr, FuncName}
{ewr, Component, FuncName}
{ewr, Component, FuncName, Params}
{ewc, A}
{ewc, Component, Params}
{ewc, Component, FuncName, Params}


So what do all those funny tuples do?

{data, Data} is simple: it tells ErlyWeb to call the corresponding view function by passing it the Data variable as a parameter, and then send result to the browser.

'ewr' stands for 'ErlyWeb redirect.' The various 'ewr' tuples simplify sending Yaws a 'redirect_local' tuple that has the URL for a component/function/parameters combination in the same app:

- {ewr, FuncName} tells ErlyWeb to return to Yaws a redirect_local to a different function in the same component.
- {ewr, Component, FuncName} tells ErlyWeb to return to Yaws a redirect_local to a function from a different component.
- {ewr, Component, FuncName, Params} tells ErlyWeb to return to Yaws a redirect_local to a component function with the given URL parameters.

For example,

{ewr, musician, list, [4]}

will result in a redirect to

http://localhost:8000/music/musician/list/4

'ewc' stands for 'ErlyWeb component.' By returning an 'ewc' tuple, you are effectively telling ErlyWeb, "render the component described by this tuple, and then send the result to the view function for additional rendering." Returning a single 'ewc' tuple is similar to 'ewr', with a few differences:

- 'ewc' doesn't trigger a browser redirect
- the result of the rendering is sent to the view function
- {ewc, Arg} lets you rewrite the arg prior to invoking other controller functions.

(If this sounds complex, don't worry -- it really isn't. Just try it yourself and see how it works.)

Now to the cool stuff: not only can your controller functions return a single 'ewc' tuple, they can also return a (nested) list of 'ewc' tuples. When this happens, ErlyWeb renders all the components in a depth-first order and the sends the final result to the view function. This lets you very easily create components that are composed of other sub-components.

For example, let's say you wanted to make blog sidebar component with several sub-components. You could implement it as follows:

sidebar_controller.erl

index(A) ->
[{ewc, about, [A]},
{ewc, projects, [A]},
{ewc, categories, [A]},
{ewc, tags, [A]}].


'sidebar_view.et'

<%@ index(Data) %>



Pretty cool, huh?

If you don't want your users to be able to access your sub-components directly by navigating to their corresponding URLs, you can implement the following function in your controllers:


private() -> true.


This will tell ErlyWeb to reject requests for private components that come from a web client directly.

Each application has one special controller that isn't part of the component framwork. This controller is always named '[AppName]\_app\_controller.erl' and it's placed in the 'src' directory. The app controller has a single function called 'hook/1', whose default implementation is


hook(A) -> {ewc, A}.


The app controller hook may return any of the values that normal controller functions return. It is useful for intercepting all requests prior to their processing, letting your rewrite the Arg or explicitly invoke other components (such as a login page).

Well, that's about it for now :) I'll appreciate any feedback, bug reports, useful code contributions, etc.

Final words

After reading all this, some of you may be thinking, "This is weird... I thought Erlang is some scary telcom thing, but what I'm actually seeing here is that Erlang is very simple... Heck, this stuff is even simpler than Rails. What's going on here?"

If that's what you're thinking, then you are right. Erlang *is* simpler than Ruby, and that's why ErlyWeb is naturally simpler than Rails. In fact, Erlang's simplicity is one of its most underrated aspects. Erlang's creators knew very well what they were doing when they insisted on keeping Erlang simple: complexity leads to bugs; bugs lead to downtime; and if there's one thing Erlangers hate the most, it's downtime.

26 comments:

Gaspar Chilingarov said...

Great, I template syntax and controller code looks a little bit spooky :P

Martin Logan said...

I have started to look into erlyweb and like what I see. I might even convert www.erlware.org over to it. Have you thought about name clashes, I can imagine a time when I might have music_view in two applications both running in the same vm - prefixes perhaps? Anyhow, nice work, and if you need a standard OTP build system to support your development I would be glad to help you with erlware - OTP Base :-)

Cheers,
Martin

Yariv said...

Martin -- I think ErlHive might be able to help with name collisions across apps running in the same VM, but I haven't experimented with ErlHive yet so I can't say for certain. Thanks for offering the build system -- maybe I'll take you up on it :)

Ulf Wiger said...

I'd love to see an ErlyWeb adaption to ErlHive. You're right in that it should solve the name clash problem. Just let me know when you want to play, and I'll do my best to assist.

Zoom.Quiet said...

Hi! i'm CPUG--ChinaPythonUserGroup 's admin.
very like ErlyWeb's gola -- simplicity, productivity and fun.

just like Pythonic!

hope get us some good example for ErlyWeb with Mnesia;
or can usage Python as view ...

i dream,can usage Python quickly develop Web Interface,and base Yaws to hold high concurrency capacity。。。

Anil said...

Going in DHH's way!

david said...

When I follow this instruction, I met some error at the initial step.
I run erlyweb:create_app("music", "/home/myid/apps")., and set the yaws.conf as follow:

docroot = /home/myid/apps/music/www
appmods =

appname = music


After I run yaws and when i put the url below in my browser, an error occurred. Here's the error. What's the matter??

Internal error, yaws code crashed

ERROR erlang code crashed:
File: appmod:0
Reason: {no_application_data,"Did you forget to call erlyweb:compile(AppDir) or add the app's previously compiled .beam files to the Erlang code path?"}
Req: {http_request,'GET',{abs_path,"/music"},{1,1}}


p.s. when i put the url as http://localhost:8000/, then it works well.

David Bergman said...

Yariv,

Your ErlyWeb is pure genius. I will test it out with a project of mine, and measure performance vs. a (good?) old LAMP version of the same app.

Alexander Fairley said...

YAWS as container.
Is anybody thinking about ways to make YAWS work as a container for other web app servers? I was playing around with Turbogears/Pylons and friends for a while, and eventually got a little irked about the scalability issues. It seems like some people are resolving this by making use of nginx as a load balancer, but it would seem to me like a bridge between YAWS and python(WSGI or what have you) might make for a nice web framework.
(I think python is an easier sell to most webdevs then erlang, but I haven't really learned erlang yet so my opinion is of course near worthless).

Yariv said...

I think Yaws/Erlang could be used to make a great reverse proxy/load balancer -- it would most certainly scale better than ones written in other languages. An Erlang load balancer combined with a Python app server sounds like a plausible setup for many webapps (although I doubt it would scale as well as an app written entirely in Erlang). But ErlyWeb for me is less about "selling" Erlang than having a blast using all its wonderful concurrent and distributed programming capabilities and functional semantics. As a developer, I find those traits more appealing than what Python has to offer, but this is to a large degree a matter of taste :)

David Haddad said...

Hey Yariv, are you planning to release the framework into the open source world? How would you like it to be extended? And do you see erlyweb as a framework to create ruby on rails type applications?

Building YAWS For Windows « :: (Bloggable a) => a -> IO () said...

[...] 21st, 2007 Inspired by Yariv’s Blog, where he talks about a framework for building web applications in Erlang, and my so far abortive [...]

Tony Perrie said...

I'm getting the following error when I try your example:

ERROR erlang code crashed:
File: appmod:0
Reason: {no_such_function,{"musician",
"index",
1,
"You tried to invoke a controller function that doesn't exist or that isn't exported"}}
Req: {http_request,'GET',{abs_path,"/music/musician"},{1,1}}

Bjorn Cintra said...

I am getting the same problem as Tony. Referring to his post here: http://thread.gmane.org/gmane.comp.lang.erlang.general/21225
I have the same setup, and everything is compiling OK, but when I try the example, it crashes with the same message. It almost seems like the erlyweb_controller.erl is nowhere to be found at runtime.

Please, any fixes?

smitty said...

fyi: for osx, the path would typically only be /usr/local if you installed from source. but the most popular means are either macports (/opt/local/) or fink (/sw/).

davber does IT » Web server performance shoot out - simple pages said...

[...] are some new hot web server frameworks: Ruby on Rails (Ruby), Yaws+ErlyWeb (Erlang) and HAppS [...]

Todd said...

Why MySQL and not Mnesia?

yudi said...

correct me if I'm wrong, but you might need to run
sudo yaws -i
?

what does erlang stand for said...

[...] language, I hope that ErlyWeb does. In fact, I don??t know of any other language that has ...http://yarivsblog.com/articles/2006/10/27/introducing-erlyweb-the-erlang-twist-on-web-framworks/What does EPMD stand for? Acronym Attic search resultWhat does EPMD stand for? abbreviation to [...]

Dmitriy said...

Hi Yariv

What if my MySQL base did not requests password:

bazil@f3t ~ $ mysql -u root
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 19
Server version: 5.0.44-log Gentoo Linux mysql-5.0.44-r1

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> \q
Bye
bazil@f3t ~ $

Erlydb can't connect to MySQL without password, but also can't with empty password:

(yaws_am_ua@f3t.dev)3> erlydb:start(mysql, [{hostname, "localhost"}, {username, "root"}, {password, ""}, {database, "music"}]).
mysql_conn:620: greeting version "5.0.44-log" (protocol 10) salt "/tz+c3tm" caps 41516 serverchar <>salt2 "Ic7(_)JyG2d#"
mysql_auth:187: mysql_auth send packet 1: <>
mysql_conn:594: init error 1045: "#28000Access denied for user 'root'@'localhost' (using password: YES)"
mysql:502: failed starting first MySQL connection handler, exiting
ok
mysql_recv:143: mysql_recv: Socket #Port closed
(yaws_am_ua@f3t.dev)4>


Could you fix it ?

Thanks.

arthur said...

when using yaws-1.77
everything works as far as ...
erlyweb:create_component("musician","/home/arthur/waps/music").
erlydb:start(mysql, [{hostname, "localhost"}, {username, "username"}, {password, "triltrom2081"}, {database, "music"}]).
erlyweb:compile("/home/arthur/waps/music",[{erlydb_driver, mysql}]).
then:
http://localhost:8000/music/musician, http://localhost:8000/musician etc all give 404

ps: the root of the app ends up localhost:800/ instead of localhost:8000/music?

tumikosha said...

Is it possible to use Erliweb with YAWS in embedded mode?
Because YAWS don't want to work on my WindowsXP in normal mode.

tumikosha said...

This blog is dead! ;0((

ubuntu下安装yaws,erlyweb成功 at 无人喝彩Beta said...

[...] 接下来建立一个测试的站点,这个按照erlyweb作者blog上的教程,在用户的home目录里建立webapp文件夹,用来存放站点文件. 按照官方说法,是在yaws shell 里输入,但是我失败了,不过我在erl里成功了,按ctrl+q后,安a确认,之后输入erl进入erlang,输入: erlyweb:create_app(”test”, “/home/kmlzkma/webapp”). [...]

ErlyWeb et Postgres « c* and code said...

[...] N’ayant pas MySQL d’installé sur ma machine, mais ayant PostgreSQL, je décide de faire tourner l’application de ce tutorial de Yariv, l’auteur d’erlyweb. [...]

Fulanpeng said...

Hi, upstairs!
I cannot read your French but I know you are talking about to use Postgresql to replace mysql.
You can try to replace the auto_increment with sequence.
After created your sequence, do not forget grant privilege to the user.
After this you can replace mysql with psql in the erlydb:start and erlydb:compile statement.

If you get success, please tell us all.

CREATE TABLE musician (
id integer primary key auto_increment,
name varchar(20),
birth_date date,
instrument enum("guitar", "piano",
"drums", "vocals"),
bio text
) type=INNODB;