17
Apr
2011
As evidence, see Scale and Scalability: Rethinking the Most Overused IT System Selling Point for the Cloud Era in IT Expert Voice.
Perhaps, as Name.com’s Sean Leach advises, there’s a lesson to be learned from Twitter: Rather than planning to scale incrementally — which, if a business finds or regains success, may now be impossible — it should plan to rework its fundamental architecture as needed, in phases, as old architectures that met earlier requirements are no longer applicable.
The coefficients of Leach’s formula may sound a bit obtuse, but in light of Twitter, perhaps the sky’s the limit. “Let’s say, the biggest you’re ever going to get is a trillion customers. But instead of designing for a trillion customers, design to a million customers so that when you get halfway there, you can redesign the system over time to be able to support a billion customers. Then when you get to almost a billion… just build it and get it out there, and then you spend your time up front and don’t worry about scaling. Plan in phases where you scale to X, and then when you get close to X, you start thinking about Y…as opposed to waiting until X happens before you worry about scaling,” he says.
The performance requirements that spawned the first generation product will impede the development of the second generation. And the best practices of just a few years ago are no longer adequate to the challenges ahead of us. This trend has only accelerated. Over the years, Twitter has replaced their original Ruby on Rails technology stack completely, and put in place a new stack based on the JVM with Scala as the primary programming metaphor.
When taking an existing system and moving it to the cloud, you must drive a number of architectural shifts as the audience grows bigger that more or less recapitulate the industry migration from the ASP model to the cloud model. For example:
- Separating the database systems and the web application so that the CPU and I/O usage of each one can be optimized separately, which leads to:
- Switching from a database instance per customer to a single database instance with sharding to reduce the cost of deployments and upgrades.
- Switching from a single database host to a database cluster to permit balancing between read and write operations; web applications are usually very read-intensive.
- Introducing multiple web application servers to permit more users to be served simultaneously, which leads to:
- Adding a caching layer in between the application and the database to reduce the amount of I/O required by each web application host, which leads to:
- Cache coordination between application hosts to further reduce the I/O to the database layer, and reduce the chance of conflicting changes to shared objects, which leads to:
- Introduction of a formal Object-Relational Layer (ORL) on top of the caching layer, to permit the web application to treat everything as a first-class object and to abstract away the work of persistance in the database.
- Separating the web application into a set of REST interfaces and a UI tier on top, permitting the same basic structure to serve web users, mobile users, business partners, your vendors.
- This leads to a realization that a relational database covered by a caching system and an ORL is actually a very expensive way to have an object-oriented or document-oriented database. If the size of the data has not already demanded the switch before this point, this is where teams realize the wisdom of CouchDB, Cassandra, HBASE, or MongoDB.
And so on. No one is making faster processors. They’re just making more of them. If your data structures and algorithms do not permit the work to be broken up and processed by an army of threads working on a grid of unremarkable machines, then you don’t have scalability. You just have size.
