Erlang Hot Code Swapping -> Hacking Nirvana

When I first heard of Erlang hot code swapping, I thought, "What a fantastic -- no, wait -- essential feature for systems that have five nines availability requirements. No wonder Erlang probably powers my phone company's 911 switch. Too bad hot I won't get to enjoy this powerful feature in my after-work Erlang hacking."

I'm happy to say I was wrong.

In my free time, I've been hacking a haXe remoting adapter into Yaws, a very powerful and scalable Erlang web server. I picked this project because think haXe is a great web client language and Erlang is unbeatable on the server side for certain purposes. I mentioned some of the reasons in previous posts and will probably discuss this more in the future (haXe is also a very good server language, by the way, and is arguably better than Erlang for many applications). What could be better than integrating the two so I can use them both in future projects?

I'm still fairly new to Erlang, and since I only work on this project on my free time, it's not going as fast as I would have wanted. Oh well.

I got the Yaws source code, and frankly I was a little lost at first. Where do I start? I decided that my first victim will be the Yaws JSON serializer/deserializer because it's an independent module. I copied json.erl to haxe.erl, opened Emacs (which I haven't used for programming since college) and a separate Erlang shell window, and modified the module's functions while testing them in the shell. That was relatively straightforward. The most challenging parts were wrapping my head around Continuation Passing Style, which the JSON parser uses, figuring out the haXe binary format, which isn't very documented, and mapping haXe types to Erlang types. haXe has class objects and Enums which Erlang doesn't have. At first, I tried to simulate classes and Enums in Erlang, but later I realized that using such structures in Erlang code would be too laborious. I decided to remove support for such types, also because I believe that arrays and anonymous objects should suffice for most RPC needs.

Now that my serializer/deserializer was finished, I got back to hacking Yaws's internals, and to my original state of confusion. I didn't know exactly how all the Yaws modules interact with each other. All I knew was that yaws_jsonrpc.erl contained the JSON RPC handling logic into which I wanted to hook. I wasn't sure how I would isolate this module from the rest of the system in order to test my implementation, which, at least initially, depended on a haXe client sending requests to the server.

My first approach was to stop Yaws, hack yaws_jsonrpc.erl (generally by adding logging statments in a few places to figure out the code flow), then run the Yaws build and install script, and restart Yaws. Needless to say, this was a very slow development effort, reminiscent of Java servlet hacking in the pre-Eclipse server integration days (a torture so horrible I wouldn't wish it upon even the new landlord who won't renew my lease :) ).

Then I had one of those earth-shattering, life-changing realizations that shook my foundations and elevated me to a higher plane of existence: This isn't Java -- it's Erlang. I can hack the code while Yaws is running and hot-deploy my changes!

Yes, it works, and it's wonderful. I run Yaws in interactive mode, where Yaws exposes an Erlang shell. Every time I make a change to a file, I simply recompile it by calling "c(FileName)." and the changes are deployed into Yaws while Yaws is running. This brings about such a speed-up in prototyping and development that any nostalgia I had left for some IDE-supported keyboard shortcut for a maddeningly slow server restart has gone in a puff of smoke.

This hot code-deploy trick is probably old news to experienced Erlang hackers, but for me it was exciting. Now that I'm armed with new knowledge, my challenge is to stop blogging about coding and to actually write code so I can get this haXe remoting adapter finished.