Monday, August 14, 2006

Smerl: Simple Metaprogramming for Erlang

Quick download link: smerl.erl

Update (8/14/06, 11:11PM ET): I added to smerl.erl the function has_func, which queries whether a module has a given function.

Update (8/14/06, 11:24PM ET): Take that, Ivan the Mean Java Dev! :)

Update (8/14/06, 11:56PM ET): A few minor bug fixes

Update (8/15/06, 14:55PM ET): Added smerl:for_file and wrote a follow-up posting.

Update (8/16/06, 23:30PM ET): Added smerl:get_func, which retrieves the abstract form for a function, and smerl:replace_func, which calls smerl:remove_func followed by smerl:add_func. Also wrote another follow-up article.

Update (8/17/06): Smerl now supports fun expressions! Read my follow-up article here.

Update (8/18/06): Added new functions, smerl:get_module, smerl:get_forms and smerl:get_exports. Also, the last update turns out to be less exciting than it sounds: fun expressions are only supported in the Erlang shell, at least for now.

Update (8/19/06): Big breakthrough: metacurrying! Read this follow-up article.

I posted a couple of weeks ago a short article discussing some thoughts I had about Metaprogramming with Erlang. It turns out that Erlang's dynamic typing, hot code swapping and built-in parsing and compilation tools make for some excellent runtime metaprogramming capabilities (I actually don't know of any other functional language that gives the programmer so much flexibility in the area of runtime code manipulation). The only problem is that using this capability is far from obvious and I doubt many programmers are aware of it. I was fortunate enough to discover it when I was digging through Ulf Wiger's rdbms_codegen.erl module in Jungerl, but otherwise it might have flown past my radar.

I wanted to make life easy for myself and my fellow Erlang (meta)programmers who want to take advantage of this capability, so I created created a small library called Smerl: Simple Metaprogramming for Erlang.

Following is the main documentation for Smerl, taken from smerl.erl (additional documentation is in the source).

Smerl is an Erlang library that simplifies the creation and manipulation of Erlang modules in runtime.

Smerl uses Erlang's capabilities for hot code swapping and abstract syntax tree parsing to do its magic. Smerl is inspired by the rdbms_codegen.erl module in the RDBMS application written by Ulf Wiger. RDBMS is part of Jungerl.

Here's a quick example illustrating how to use Smerl:

test_smerl() ->
C1 = smerl:new(foo),
{ok, C2} = smerl:add_func(C1, "bar() -> 1 + 1."),
foo:bar(). % returns 2

New functions can be expressed either as strings of Erlang code or as abstract forms. For more information, read the Abstract Format section in the ERTS User's guide (link).

Using the abstract format, the 3rd line of the above example would be written as

{ok,C2} = smerl:add_func(C1, {function,1,bar,0,

The abstact format may look more verbose in this example, but it's also more amenable to runtime manipulation.

To manipulate or query an existing module rather than a new module, the first line could be rewritten such as:

C1 = smerl:for_module(mnesia),

Detailed Description

With Smerl, you can both create new modules and manipulate existing modules in runtime. You can also query whether a module has a given function by calling smerl:has_func. To start creating a new module, call smerl:new(ModuleName). To start modifying an existing module, call smerl:for_module(ModuleName). (The module be accessible with code:which and either have been compiled debug_info or its source file must in the same directory as the .beam file or in a ../src directory relative to the .beam file's ./ebin directory.) Both these functions return an opaque context record for the module. To manipulate the module, use smerl:add_func and smerl:remove_func. Just remember not to add the same function name with the same arity twice as it will eventually result in a compilation error.

When you're ready to compile your module, call smerl:compile, passing in the opaque context record. If there are no errors, you can start using the new module.

If you have any problems or suggestions about Smerl, please let me know.

Smerl isn't just a programming exercise. It may not be obvious, but Smerl actually has very good uses. I will write more about Smerl soon.

New tip (8/18/06): Are you looking an easy way of knowing what the abstract form for a function is? Fire up the erlang shell and type

io:parse_erl_form(". ").

This will give you a one-line mini shell, in which you can type your function and get back its abstract form representation.


polli said...

This could be fun :) Have you used it somewhere in a "real" situtations?

Yariv said...

I'm actually using it right now as part of a larger web framework. Stay tuned because good things are coming :)

erlangist said...

Let me guess... You are implementing Erlang on rails!

Moe Aboulkheir said...

Regarding your comment about unparalleled flexibility: Scheme is an impure functional language, and being a dialect of Lisp, syntactic abstraction and metaprogramming are core features.

Yariv said...

Hi Moe, thanks for the feedback. I haven't used Scheme in years, so I guess I made a mistake by saying that Lisp is a pure functional language. I know Lisp has great metaprogramming capabilities -- at least for compile-time macroes -- but I don't know much about its runtime metaprogramming capabilities. Thanks for the pointer -- I will research it further.

Yariv said...

Erlangist -- I actually prefer to think of it is a Erlang on Jet Engines or something, but yeah, you get the point :)

Neville said...

Hi Yariv,

One way of hyping Erlang to the web community would be to build a Tadalist like app to compare with the Ruby on Rails version, and the Rife [Java etc] version,

That would really put the Erlang cat amongst the pidgeons!

Yariv said...

Neville, keep reading this blog, and you'll see some cool stuff -- not just by me, but also other people. Thanks for the feedback!

Danno said...

Bah, Erlang's too powahful for simple Todo list applications.

Howabout something that's cool and useful and really shows off its concurrent muscles.

Neville said...

Danno, maybe so.

But BlaBlaList generated a *lot* of discussion when it took on Tadalist.

And a web app is going to flex Erlang's concurrent muscles, not to mention showcasing haXe, Smerl, Yaws etc.

Yariv said...

I know of a few people who are building webapps in Erlang that will be quite cutting edge. I will try to build one too in the near future. I totally agree there's no better way to demonstrate the benefits of Yaws than to show a killer app written in Erlang.

lambder » Erlang’s Dynamism said...

[...] can acheive the p.3 result using a helper tools such as LFE, Smerl or  “Dynamic” module generation with compile-time macros Categories: [...]