Scott Rosenberg makes an interesting observation about Web 2.0 software. Scott wrote the book Dreaming in Code, which gives a look into the making of the open source software project Chandler. One of the key points Scott makes in his book is that software is hard. One of the most common arguments against this point is that web applications solve this problem. Web applications are agile in their nature. They can be rapidly developed, deployed and continually updated with new features and fixes.
Web 2.0 makes software almost easy in comparison to past software revolutions. Not quite. As Scott points out in his blog post, both Yahoo! and Bloglines have taken nearly five years to release new versions of their web (2.0) applications — Yahoo! Mail and Bloglines’ namesake app. Five years is what it took Microsoft to get Vista out the door. While it’s easy to build web applications and services in a rapid manner, it’s not necessarily easier to make them scale. I pointed out yesterday that Twitter is having issues with scaling their service as it grows ever more popular. Facebook, MySpace, and many other Web 2.0 companies have faced the same challenge.
I don’t think many people touting Web 2.0’s software development superiority often understand what is involved in making a scalable web app/service. There are so many pieces involved and just addressing some of them (say the database layer or client optimizations) is not an option. Once one piece is made more scalable you find the other pieces are hindering the performance and stability of the application/service.
I believe that the next revolution for Web 2.0 (I think I’ll call this Web 2.1) is allowing developers to build on top of an infrastructure that handles the core scaling gotchas from the start transparently, without large up-front capital investment. Amazon.com’s AWS offerings appear to be headed in this direction. They’re still missing some key pieces (a reliable/persistent data store & load balancing for example), but it’s promising. The ideal would be for developers to not have to worry so much about the underlying hardware, network, operating systems, etc. For example, if you’re building the next great web app/service with Ruby on Rails, you would love to be able to deploy in an environment that allows you to grow without any worries. You’d want to know that you have things like:
- Nearly endless data storage that is reliable and responsive
- As much bandwidth as necessary (throttle on demand)
- A data store that grows without concerns of having to purchase additional hardware and software
- An authentication & authorization service
- A standards based reliable messaging service
- The ability to seamlessly add more instances in the web and app tiers (on demand)
- A caching service for all levels of caching (client, database, external data sources, etc.)
If you can have those types of items covered in your deployment environment up-front without the typical capital investments (a big gotcha for today’s Web 2.0 companies, by the way), then I think the five year development cycle Scott Rosenberg references with the latest Yahoo! Mail and Bloglines’ releases can be lessened drastically. The problem today is that you can build your web app/service with scaling in mind but it takes substantial amounts of time to ramp up and test for each increment in the underlying infrastructure. If Twitter (for example) started out running on an infrastructure that had the items I listed above, I think they’d be able to better handle the exponential growth they’ve been experiencing. They didn’t start out on an infrastructure like that because the capital costs are too much initially.
Bottom line, Web 2.0 software development is not a panacea. (Ditto for SOA, as Jim Stogdill (Gestalt’s CTO) pointed out previously.) There are still many challenges faced by Web 2.0 software that makes it far from ideal in terms of solving the age old problems inherent in software. The good news is that there are opportunities to help solve these problems. The question I have now is: Who will step up to the challenge?