Azure, don't give me multiple VMs, give me one elastic VM

Yesterday, Microsoft revealed new major features for Windows Azure (see ScottGu's post). It all looks shiny and great, but after reading most of the material describing the new features, I still find the overall idea behind all of it flawed: why should I care on how much VMs my web app runs? Isn't that a problem to solve for the Windows Azure engineers / software? And what if I need the file system, why can't I simply get a virtual filesystem ?

To illustrate my point, let's use a real example: a product website with a customer system/database and next to it a support site with accompanying database. Both are written in .NET, using ASP.NET and use a SQL Server database each. The product website offers files to download by customers, very simple.

You have a couple of options to host these websites:

  • Buy a server, place it in a rack at an ISP and run the sites on that server
  • Use 'shared hosting' with an ISP, which means your sites' appdomains are running on the same machine, as well as the files stored, and the databases are hosted in the same server as the other shared databases.
  • Hire a VM, install your OS of choice at an ISP, and host the sites on that VM, basically the same as the first option, except you don't have a physical server
  • At some cloud-vendor, either host the sites 'shared' or in a VM. See above.

With all of those options, scalability is a problem, even the cloud-based ones, though not due to the same reasons:

  • The physical server solution has the obvious problem that if you need more power, you need to buy a bigger server or more servers which requires you to add replication and other overhead
  • Shared hosting solutions are almost always capped on memory usage / traffic and database size: if your sites get too big, you have to move out of the shared hosting environment and start over with one of the other solutions
  • The VM solution, be it a VM at an ISP or 'in the cloud' at e.g. Windows Azure or Amazon, in theory allows scaling out by simply instantiating more VMs, however that too introduces the same overhead problems as with the physical servers: suddenly more than 1 instance runs your sites.

If a cloud vendor offers its services in the form of VMs, you won't gain much over having a VM at some ISP: the main problems you have to work around are still there: when you spin up more than one VM, your application must be completely stateless at any moment, including the DB sub system, because what's in memory in instance 1 might not be in memory in instance 2.

This might sounds trivial but it's not. A lot of the websites out there started rather small: they were perfectly runnable on a single machine with normal memory and CPU power. After all, you don't need a big machine to run a website with even thousands of users a day. Moving these sites to a multi-VM environment will cause a problem: all the in-memory state they use, all the multi-page transitions they use while keeping state across the transition, they can't do that anymore like they did that on a single machine: state is something of the past, you have to store every byte of state in either a DB or in a viewstate or in a cookie somewhere so with the next request, all state information is available through the request, as nothing is kept in-memory.

Our example uses a bunch of files in a file system. Using multiple VMs will require that these files move to a cloud storage system which is mounted in each VM so we don't have to store the files on each VM. This might require different file paths, but this change should be minor. What's perhaps less minor is the maintenance procedure in place on the new type of cloud storage used: instead of ftp-ing into a VM, you might have to update the files using different ways / tools.

All in all this makes moving an existing website which was written for an environment that's based around a VM (namely .NET with its CLR) overly cumbersome and problematic: it forces you to refactor your website system to be able to be used 'in the cloud', which is caused by the limited way how e.g. Windows Azure offers its cloud services: in blocks of VMs.

Offer a scalable, flexible VM which extends with my needs

Instead, cloud vendors should offer simply one VM to me. On that VM I run the websites, store my DB and my files. As it's a virtual machine, how this machine is actually ran on physical hardware (e.g. partitioned), I don't care, as that's the problem for the cloud vendor to solve. If I need more resources, e.g. I have more traffic to my server, way more visitors per day, the VM stretches, like I bought a bigger box. This frees me from the problem which comes with multiple VMs: I don't have any refactoring to do at all: I can simply build my website as if it runs on my local hardware server, upload it to the VM offered by the cloud vendor, install it on the VM and I'm done.

"But that might require changes to windows!"
Yes, but Microsoft is Windows. Windows Azure is their service, they can make whatever change to what they offer to make it look like it's windows. Yet, they're stuck, like Amazon, in thinking in VMs, which forces developers to 'think ahead' and gamble whether they would need to migrate to a cloud with multiple VMs in the future or not. Which comes down to: gamble whether they should invest time in code / architecture which they might never need. (YAGNI anyone?)

So the VM we're talking about, is that a low-level VM which runs a guest OS, or is that VM a different kind of VM?

The flexible VM: .NET's CLR ?

My example websites are ASP.NET based, which means they run inside a .NET appdomain, on the .NET CLR, which is a VM. The only physical OS resource the sites need is the file system, however this too is accessed through .NET. In short: all the websites see is what .NET allows the websites to see, the world as the websites know it is what .NET shows them and lets them access. How the .NET appdomain is run physically, that's the concern of .NET, not mine. This begs the question why Windows Azure doesn't offer virtual appdomains? Or better: .NET environments which look like one machine but could be physically multiple machines.

In such an environment, no change has to be made to the websites to migrate them from a local machine or own server to the cloud to get proper scaling: the .NET VM will simply scale with the need: more memory needed, more CPU power needed, it stretches. What it offers to the application running inside the appdomain is simply increasing, but not fragmented: all resources are available to the application: this means that the problem of how to scale is back to where it should be: with the cloud vendor.

"Yeah, great, but what about the databases?"
The .NET application communicates with the database server through a .NET ADO.NET provider. Where the database is located is not a problem of the appdomain: the ADO.NET provider has to solve that. I.o.w.: we can host the databases in an environment which offers itself as a single resource and is accessible through one connection string without replication overhead on the outside, and use that environment inside the .NET VM as if it was a single DB.

But what about memory replication and other problems?

This environment isn't simple, at least not for the cloud vendor. But it is simple for the customer who wants to run his sites in that cloud: no work needed. No refactoring needed of existing code. Upload it, run it.

Perhaps I'm dreaming and what I described above isn't possible. Yet, I think if cloud vendors don't move into that direction, what they're offering isn't interesting: it doesn't solve a problem at all, it simply offers a way to instantiate more VMs with the guest OS of choice at the cost of me needing to refactor my website code so it can run in the straight jacket form factor dictated by the cloud vendor.

Let's not kid ourselves here: most of us developers will never build a website which needs a truck load of VMs to run it: almost all websites created by developers can run on just a few VMs at most. Yet, the most expensive change is right at the start: moving from one to two VMs. As soon as you have refactored your website code to run across multiple VMs, adding another one is just as easy as clicking a mouse button. But that first step, that's the problem here and as it's right there at the beginning of scaling the website, it's particularly strange that cloud vendors refuse to solve that problem and leave it to the developers to solve that.

Which makes migrating 'to the cloud' particularly expensive.

7 Comments

  • This isn't a new idea, and is exactly what Platform-as-a-Service cloud vendors aim at (e.g. Heroku, AppHarbor, Google AppEngine, Nodejitsu, etc, etc).

    They're mostly pretty successful at it.

    The problem w/ Azure (imo) is that they can't decide if they want to be a PaaS or an IaaS (infrastructure-as-a-service, a la Amazon Web Services).

  • You really want one giant VM? It sounds like you want the abstraction at the OS or at the .Net framework level to provide you a way to transparently scale out your applications. That seems like the wrong point to abstract/scale. It seems like the abstraction should be at the application level, not at the OS level.

  • You are missing a big aspect of cloud computing which us availability. If your one giant VM has any issue, your site is down.

  • Interesting... two thoughts popped into my head which I wanted to share:

    1. One day we will write apps so that they can scale, it's just a transitional phase now. So today we don't see a benefit, but tomorrow we don't see any downside.

    2. Or that is not feasible, just like programming with multiple cores turns out to be hard. So we will invent ways to hide complexity. Man is seemingly not able to wrap its head around such complexities, so the very smartest will have to abstract it away. 1 app domain, 1 connection string, one in memory cache, 1 core, that's the mental model for devs. And yet, it runs massively parallel, even when running locally on our test machine.

  • I agree with Ayende.

  • As to the "availability" post about the one giant VM being down - I think the point was that again falls to the cloud vendor to make what looks to the customer as "one giant VM" is in reality many servers in many locations, so availability is no more a problem than it is with multiple VMs. If hardware fails, or even OS fails, the underlying cloud architecture would spin up new hardware, etc to keep the application running (of course, if the application itself is crap, then not much the cloud provider can do - but that is a different availability problem).

  • Azure has the PaaS offer, which provides elastic computing, that's pretty much like what you are looking for. Where you can instruct it to instantiate as many Web Rioles and Worker Roles as the load demands.

    On the other hand IaaS is about hosting your VMs as it is, and you have the complete control, and the mess is yours to handle :)

    In order not to do much work to migrate your web-sites to Azure IaaS, I believe you can have a load balancer which directs requests of the same session to the same VM instance, so you can rely on VM memory for data caching.

Comments have been disabled for this content.