True clouds need automated configuration and deployment

Rachel Chalmers & Csilla Zsigri, The 451 Group

Like large-scale virtual infrastructure before it, the true cloud will need automation to manage the complexity and scale of the configuration and deployment work required. Unfortunately, the term 'automation' is used to cover almost as bewildering a range of technical approaches as the term 'virtualization'. Before we can have a sensible conversation about automation in the context of emerging public and private cloud computing, we need to take a step back and talk about what automation actually is.

Automation as we know it today grew out of early experiments in the cloud. Back in those days, though, we called public clouds ASPs, a term that fell into well-deserved disrepute. Today's automation frameworks trace their lineage to two ASPs in particular: Breakaway Solutions and Loudcloud. At the turn of the millennium, during the dot-com boom, Breakaway employed Dev Ittycheria and Vijay Manwani, and Loudcloud was the second go-round for Netscape wunderkind Marc Andreessen.

As their successors have discovered, building application hosting infrastructure to meet Internet-scale demand was no easy undertaking. In fact, it was the failure of ASPs to scale elegantly that tarnished their acronym and their model. Ittycheria, Manwani and Andreessen identified one of the bottlenecks involved. It was a labor-intensive and wearisome task to get fresh x86 servers off a truck and provision them from the bare metal with the operating system and applications they would need to run in production. The work was boring and repetitive, and therefore error-prone. It was exactly the kind of dull chore best handled by a machine, thereby freeing overworked systems administrators for work that might actually contribute to a company's bottom line.

The three entrepreneurs sought to address this problem in different, though related, ways. Ittycheria and Manwani bought Network Shell, which promised Unix-like administrative power over Windows boxes, and rechristened it BladeLogic. Andreessen packaged Loudcloud's own hand-written scripts as Opsware.

BladeLogic and Opsware weren't the only provisioning automation ventures. Canadian Think Dynamics was acquired by IBM to form the basis for its Tivoli Provisioning Manager. PatchLink (now Lumension Security) bought Moonlight Systems, Sun bought Terraspring and Veritas bought both Ejasent and Jareva Technologies. What BladeLogic and Opsware became, though, were the breakout success stories in provisioning automation. When they finally sold out in 2008 and 2007, respectively, it was for about a billion dollars each.

Success spawns imitators. Entrepreneurs looked for other groups of boring chores that could be replaced by automated software processes. Among the companies that sought to reproduce the BladeLogic/Opsware approach were Enigmatec, iConclude, Opalis Software and RealOps in runbook automation; Dunes Technologies, DynamicOps, Embotics, Fortisphere, ManageIQ, Q-Layer and Reflex Systems in production VM automation; and Akimbi Systems, CollabNet, Hatsize, IT Structures, SkyTap, Surgient and VMLogix in test lab VM automation.

A good number of these have already been acquired. Opsware bought iConclude just before HP bought Opsware. BMC Software picked up both BladeLogic and RealOps. VMware acquired Dunes and Akimbi. Sun bought Q-Layer. Others we will meet again when we turn our attention to the pure cloud.

There are still companies tackling provisioning from bare metal. There are companies that assume an operating system is already in place, and concern themselves with delivering and managing resources at the application layer. Between bootstrapping the hardware and controlling the production server, however, there's still a technology gap. This is where we find companies that handle one or other of configuration and deployment automation, and right now these seem to be the hottest spots for innovation around the cloud. Companies like Enigmatec, Opalis and Surgient are at work here, in cloud server configuration automation, for example. But a lot of new companies have appeared as well.

Among the new companies on the configuration automation side are Cfengine with its Nova, DTO Solutions with ControlTier, Opscode with Chef, Reductive Labs with Puppet and RightScale. What's immediately striking about this group of companies is that, in contrast to predecessors, they are all based around open source projects. Two more open source projects, AutomateIT and bcfg2, aren't commercially sponsored yet. This state of affairs makes a critical point about cloud computing cost models: they just cannot support big software licensing taxes. This is a finding with significant negative implications for old-school proprietary software vendors like Oracle and VMware. Not surprisingly, the Sun and SpringSource acquisitions see those vendors embracing open source in new ways.

Application deployment automation comes in a number of flavors, and perhaps because it hasn't converged on a well-understood model yet, doesn't appear to be quite as vulnerable to the proliferation of open source alternatives. Companies here include Eucalyptus (which actually is open source), CohesiveFT with its Elastic Server product, Nolio, Phurnace Software, rPath, Vubble and webappVM.

Lots of other new ventures hope to claim their own automation territory as well. Doyenz wants to automate cloud backup and disaster recovery; LineSider Technologies offers configuration automation at the network layer. RNA Networks says its memory virtualization is effectively an automation layer between the server and storage tiers; Stratavia calls its tools database automation. Platform-as-a-service (PaaS) clouds from Microsoft Azure and Google App Engine to Engine Yard and Heroku come with a substantial amount of automation technology built in. It's part of the package: cloud is, essentially, IT as a service, made possible by the combination of virtualization and automation.

Just as no incumbent vendor can afford to be without a coherent virtualization strategy, none can afford to be without some skin in the automation game. Virtualization and automation are preconditions for the cloud, but on their own, they are not enough to qualify a company as a trusted cloud vendor.

Who will fish in the cloud automation pond? Look for opportunistic tuck-in acquisitions from the old enterprise systems management vendors (BMC, CA, HP and IBM), networking vendors (Cisco Systems), virtualization management companies (Citrix Systems, Microsoft, Novell, Parallels, Red Hat and VMware) and emerging PaaS titans (Google with its enterprise ambitions, IBM with WebSphere, Microsoft with Azure and SharePoint, Oracle with WebLogic, Red Hat with JBoss, SAP, salesforce.com and VMware with SpringSource).