Sunday, April 13, 2008

EC2 gets persistent block level storage

I just caught Amazon's announcement of the new persistent storage engine for EC2. This is great stuff. It lets you create persistent block level storage devices ranging from 1GB to 1TB in size and attach them to EC2 instances in predetermined availability zones. This service complements Amazon's other storage services -- EC2 and SimpleDB -- in providing raw block-level storage devices that are persistent, fast and local (so you don't have to worry about SimpleDB's eventual consistency issues). You can use these volumes for anything -- running a traditional DBMS (MySQL, Postgres) is the first thing that comes to mind.

This announcement is a departure from Amazon's tradition of announcing services only once they become available. It looks like Amazon is feeling the heat of competition from Google App Engine and is becoming more open to win over the hearts and minds of developers who are drawn to GAE for its auto-magical scalability. The ability to attach multiple terabyte-sized volumes on demand alleviates some of those concerns when deploying on Amazon's infrastructure. I'm sure it won't be long before someone creates an open source BigTable-like solution for applications that need massive scalability and redundancy on top of multiple persistent storage volumes (I think this would be a great application to write in Erlang, but I don't know how well Erlang performs in applications that require heavy disc IO).

I like what Amazon is doing. By providing the basic building blocks for scalable applications, its enables startups to create their own GAE competitors (Heroku is the first one that comes to mind) on top of Amazon's infrastructure. Smart move.

Google has the advantage of being able to provide APIs for tight integration with other Google services such as authentication and search (the latter is hypothetical as of now). We'll see how strongly this plays in Google's favor in the coming months.

Of course, price is still a question mark. Neither Amazon's persistent storage service nor GAE have had their prices announced.

Another missing detail is the Amazon store service's reliability. If a disc fails, do you lose your data? What's the failure probability? Etc.

All this is great for developers. Competition between Amazon and Google means developers will enjoy more services and for lower prices in the coming years.


Thorsten - CTO RightScale said...

Having tested the storage volumes for a while I have to say they're great! I wrote some more about how it changes the game at
The Amazon folks are on a roll!

With the addition of the storage volumes there's no doubt in my mind anymore: the cloud adopters will have much more computing horsepower and flexibility at their fingertips than those who are still racking their own machines. Cloud computing is going to be as significant for deployment as agile is for software development. You either compute in the cloud or you'll be left behind by your competitors because they can deploy faster, better, and cheaper than you can.

nosewheelie » Blog Archive » Persistent block level storage in EC2 said...

[...] Yariv: I just caught Amazon’s announcement of the new persistent storage engine for EC2. This is great [...]

Akshay Surve said...

EC2 will get persistent block level storage ;) Currently, its in closed beta.