by Andrew Oram
American Reporter Correspondent
December 20, 2010
WHY CLOUDS AND WEB SERVICES WILL TAKE OVER COMPUTING
CAMBRIDGE, Mass. -- The tech press is intensely occupied and pre-occupied with analyzing the cloud from a business point of view. Should you host your operations in a cloud provider? Should you use web services for office work? The stream of articles and blogs on these subjects shows how the cloud is indisputably poised to take over the real world of computing.
But the actual conclusions these analysts reach are intensely conservative: watch out, count up your costs carefully, look closely at regulations and liability issues that hold you back, etc. The analysts are obsessed with the cloud, but they're not encouraging companies to actually use it - or at least they're saying we'd better put lots of thought into it first.
My long-term view convinces me we all WILL be in the cloud. No hope in bucking the trend. The advantages are just too compelling.
I won't try to replicate here the hundreds and hundreds of arguments and statistics produced by the analysts. I'll just run quickly over the pros and cons of using cloud computing and web services, and why they add up to a ringing endorsement. That will help me get to the question that really concerns this article: what can we do to preserve freedom in the cloud?
The promise of the cloud shines bright in many projections. The federal government has committed to a "Cloud First" policy in its recent Information Technology reform plan. The companies offering IaaS, and Paas, and SaaS promulgate mouth-watering visions of their benefits. But some of the advantages I see aren't even in the marketing literature - and some of them, I bet, could make even a free software advocate come around to appreciating the cloud.
The standard litany of reasons for moving to IaaS or PaaS can be summarized under a few categories:
Low maintenance: No more machine rooms, no more disk failures (that is, disk failures you know about and have to deal with), no more late-night calls to go in and reboot a critical server.
These simplifications, despite the fears of some Information Technology professionals, don't mean companies can fire their system administrators. The cloud still calls for plenty of care and feeding. Virtual systems go down at least as often as physical ones, and while the right way to deal with system failures is to automate recovery, that takes sophisticated administrators.
So the system administrators will stay employed and will adapt. The biggest change will be a shift from physical system management to diddling with software; for an amusing perspective on the shift see my short story Hardware Guy.
Fast ramp-up and elasticity: To start up a new operation, you no longer have to wait for hardware to arrive and then lose yourself in snaking cables for hours. Just ask the cloud center to spin up as many virtual systems as you want.
Innovative programmers can also bypass IT management, developing new products in the cloud. Developers worry constantly whether their testing adequately reproduces the real-life environment in which production systems will run, but if both the test systems and the final production systems run in the cloud, the test systems can match the production ones much more closely.
Because existing companies have hardware and systems for buying hardware in place already, current cloud users tend to come from high-tech start-ups. But any company that wants to launch a new project can benefit from the cloud. Peaks and troughs in usage can also be handled by starting and stopping virtual systems - you just have to watch how many get started up, because a lack of oversight can incur run-away server launches and high costs.
Cost savings: In theory, clouds provide economies of scale that undercut anything an individual client could do on their own. How can a private site, chugging away on a few computers, be more efficient than thousands of fungible processors in one room under the eye of a highly trained expert, all strategically located in an area with cheap real estate and electricity?
Currently, the cost factor in the equation is not so cut and dried. Running multiple servers on a single microprocessor certainly brings savings, although loads have to be balanced carefully to avoid slowing down performance unacceptably. But running processors constantly generates heat, and if enough of them are jammed together the costs of air conditioning could exceed the costs of the computers. Remote computing also entails networking costs.
It will not take long, however, for the research applied by cloud vendors to pay off in immense efficiencies that will make it hard for organizations to justify buying their own computers.
Elasticity and consolidation make IaaS so attractive that large companies are trying to build "private clouds" and bring all the organization's server hardware into one department, where the hardware is allocated as virtual resources to the rest of the company. These internal virtualization projects don't incur some of the disadvantages that this article addresses, so I won't consider them further.
SaaS offers some benefits similar to IaaS and PaaS, but also significant differences.
Low maintenance: No more installation, no more upgrades, no more incompatibilities with other system components or with older versions of the software on other people's systems. Companies licensing data, instead of just buying it on disks, can access it directly from the vendor's site and be sure of always getting the most recent information.
Fast ramp-up and elasticity: As with IaaS, SaaS frees staff from running every innovation past the IT group. They can recreate their jobs and workflows in the manner they want.
Feedback: To see what's popular and to prioritize future work, companies love to know how many people are using a feature and how long they spend in various product functions. SaaS makes this easy to track because it can log every mouse click.
Enough of the conventional assessment. What hidden advantages lie in clouds and web services?
What particularly should entice free and open software software advocates is web services' prospects for making money. Although free software doesn't have to be offered cost-free (as frequently assumed by those who don't know the field), there's no way to prevent people from downloading and installing it, so most of the money in free software is made through consulting and additional services.
Web services allow subscriptions instead, a much more stable income. Two popular content management systems exemplify this benefit: WordPress offers hosting at wordpress.com and Drupal at drupalgardens.com, all while offering their software as open source.
But I find another advantage to web services. They're making applications better than they ever have been in the sixty-year history of application development.
Compare your own experiences with stand-alone software to websites. The quality of the visitor's experience on a successful website is much better. It's reminiscent of the old cliché about restaurant service in capitalist versus socialist economies.
According to this old story, restaurants in capitalist countries depend on repeat business from you and your friends, driving the concern for delivering a positive customer experience from management down to the lowest level of the wait staff. In a socialist economy, supposedly, the waiters know they will get paid whether you like their service or not, so they just may not try very hard. Perhaps taking pains to make you happy would be degrading to them as heroes of a workers' society.
I don't know whether this phenomenon is actually true of restaurants, but an analogous dynamic holds in software. Web sites know that visitors will vanish in half a second if the experience is not immediately gripping, gratifying, and productive. Every hour of every day, the staff concentrate on the performance and usability of the site. Along with the business pressure on web services to keep users on the page, the programmers there can benefit from detailed feedback about which pages are visited, in which order, and for how long.
In contrast, the programmers of stand-alone software measure their personal satisfaction by the implementation of complex and sophisticated calculations under the product's surface. Creating the user interface is a chore relegated to less knowledgeable staff.
Whatever the reason, I find the interfaces of proprietary as well as free software to be execrable, and while I don't have statistics to bolster my claim. I think most readers can cite similar experiences. Games are the main exception, as well as a few outstanding consumer applications, but these unfortunately do not seem a standard for the vast hoards of other programmers to follow.
Moving one's aching fingers from stand-alone software to a web service brings a sudden rush of pleasure, affirming what working with computers can be. A bit of discipline in the web services world would be a good cold bath for the vendors and coders.
So why are the analysts and customers still wary of cloud computing? They have their reasons, but some dangers are exaggerated.
Managers responsible for sensitive data feel a visceral sense of vulnerability when they entrust that data to some other organization. Web services have indeed had breaches, because they are prisoners of the twin invariants that continue to ensure software flaws: programmers are human, and so are administrators. Another risk comes when data is transmitted to a service such as Amazon.com's S3, a process during which it be seen or even in theory altered.
Still, I expect the administrators of web and cloud services to be better trained and more zealous in guarding against security breaches than the average system administrator at a private site. The extra layer added by IaaS also creates new possibilities. An article called "Security in the Cloud" by Gary Anthes, published in the November 2010 Communications of the ACM, points to research projects by Hewlett-Packard and IBM that would let physical machines monitor the virtual machines running on them for viruses and other breaches of security, a bit like a projectionist can interrupt a movie.
A cloud or web service provider creates some risk just because it provides a tasty target to intruders, who know they can find thousands of victims in one place. On the other hand, if you put your data in the cloud, you aren't as likely to lose it to some drive-by trouble-seeker picking it up off of a wireless network that your administrator failed to secure adequately, as famously happened to T.J. Maxx (and they weren't alone).
And considering that security experts suspect most data breaches to be internal, putting data in the cloud might make it more secure by reducing its exposure to employees outside of the few programmers or administrators with access rights. If the Department of Defense had more systems in the cloud, perhaps it wouldn't have suffered such a sinister security breach in 2008 through a flash drive with a virus.
In general, the solution to securing data and transactions is to encrypt everything. Encrypting the operating systems loaded in IaaS, for instance, gives the client some assurance that no one can figure out what it's doing in the cloud, even if another client or even the vendor itself tries to snoop. If some technological earthquake undermines the integrity of encryption technologies - such as the development of a viable quantum computer - we'll have to rethink the foundations of the information age entirely anyway.
The main thing to remember is that most data breaches are caused by lapses totally unrelated to how servers are provisioned: they happen because staff stored unencrypted data on laptops or mobile devices; because intruders slipped into applications by exploiting buffer overflows or SQL injection; and so on. (See, for instance, a U.S. Health & Human Services study saying that "Laptop theft is the most prevalent cause of the breach of health information affecting more than 500 people.)
Regulations such as HIPAA can rule out storing some data off-site, and concerns about violating security regulations come up regularly during cloud discussions. But these regulations affect only a small amount of the data and computer operations, and the regulations can be changed once the computer industry shows that clouds are both valuable and acceptably secure.
Bandwidth is a concern, particularly in less technologically developed parts of the world (like much of the United States, come to think of it), where bandwidth is inadequate. But in many of these areas, people often don't even possess computers. SaaS is playing a major role in underdeveloped areas because it leverages the one type of computer in widespread use (the cell phone) and the one digital network that's widely available (the cellular grid). So in some ways, SaaS is even more valuable in underdeveloped areas, just in a different form from regions with high bandwidth and universal access.
Nevertheless, important risks and disadvantages have been identified in clouds and web services. IaaS and PaaS are still young enough (and their target customers sophisticated enough) for the debate to keep up pretty well with trends; in contrast, SaaS has been crying out quite a while for remedies to be proposed, such as the Recommendations for Best Practicesrecently released by the Consumer Federation of America. This article will try to elevate those questions to the next level, to find more lasting solutions to the following problems:
Availability: Every system has down time, but no company wants to be at the mercy of a provider that turns off service, perhaps for 24 hours or more, because they failed to catch a bug in their latest version or provide adequate battery backup during a power failure.
When WikiLeaks was forced off of Amazon.com's cloud service, it sparked outrage whose echo reached as far as a Wall Street Journal blog and highlighted the vulnerability of depending on clouds. Similarly, the terms of service on social networks and other SaaS sites alienate some people who feel they have legitimate content that doesn't pass muster on those sites.
Liability: One of the big debates in the legal arena is how to apportion blame when a breach or failure happens in a cascading service, where one company leases virtual systems in the cloud to provide a higher-level service to other companies.
Reliability: How can you tell whether the calculation that a service ran over your corporate data produced the correct result? This is a lasting problem with proprietary software, which the free software developers argue they've solved, but which most customers of proprietary software have learned to live with and which therefore doesn't turn them against web services.
But upgrades can present a problem. When a new version of stand-alone software comes out, typical consumers just click "Yes" on the upgrade screen and live with the consequences. Careful system administrators test the upgrade first, even though the vendor has tested it, in case it interacts perniciously with some factor on the local site and reveals a bug. Web services reduce everyone to the level of a passive consumer by upgrading their software silently. There's no recourse for clients left in the lurch.
Control: Leaving the software on the web service's site also removes all end-user choice. Some customers of stand-alone software choose to leave old versions in place because the new version removed a feature the customers found crucial, or perhaps just because they didn't want the features in the new version and found its performance worse. Web services offer one size to fit all.
Because SaaS is a black box, and one that can change behavior without warning to the visitors, it can provoke concerns among people sensitive about consistency and reliability. See my article Results from Wolfram Alpha:. All the Questions We Ever Wanted to Ask About Software as a Service.
Privacy: Web services have been known to mine customer data and track customer behavior for marketing purposes, and have given data to law enforcement authorities. It's much easier to monitor millions of BlackBerry messages traveling through a single server maintained by the provider than the messages bouncing in arbitrary fashion among thousands of Sendmail servers. If a customer keeps the data on its own systems, law enforcement can still subpoena it, but at least the customer knows she's being investigated.
In the United States, furthermore, the legal requirements that investigators must meet to get data is higher for customers' systems than for data stored on a third-party site such as a web service. Recent Congressional hearings (discussed on O'Reilly's Radar site highlighted the need to update U.S. laws to ensure privacy for cloud users.
These are knotty problems, but one practice could tease them apart: making the software running clouds or web services open source.
A number of proponents for this viewpoint can be found, such as the Total Information Outsourcing group, as well as a few precedents.
Besides the WordPress and Drupal services mentioned earlier, StatusNet runs the microblogging site identi.ca and opens up its code so that other people could run sites that interoperate with it. Source code for Google's AppEngine, mentioned earlier as a leading form of IaaS, has been offered for download by Google under a free license. Talend offers data integration and business intelligence as both free software and SaaS.
The Free Software Foundation, a leading free software organization that provides a huge amount of valuable software to Linux and other systems through the GNU project, has created a license called the GNU Affero General Public License that encourages open code for web services. When sites such as StatusNet release code under that license, other people are free to build web services on it but must release all their enhancements and bug fixes to the world as well.
What problems can be ameliorated by freeing the cloud and web service software? Can the companies who produced that software be persuaded to loosen their grip on the source code? And what could a world of free cloud and web services look like? That is where we will turn next.>p> Next: Why web services should be released as free software.
Editor's Note: Previous installments of this series (Introduction, followed by Parts 1, 2, 3 ands 4) are available as and in the order they were published from the drop-down menu at top left of this page. Just click on ANDY ORAM REPORTS.