Development for Scalability
When you launch your website, product or app, how do you ensure that it performs just as well with a handful of users as it does with thousands or millions? Several mechanisms can be employed such as caching, increasing the specification of servers, separating architecture layers to scale each as required. Each of these comes with its drawbacks.
Caching involves taking a snapshot of a webpage or piece of data and serving that to users instead of generating the response from a database. This reduces load on the server and provides an almost immediate result when requested. But data soon becomes stale. How do you know when to throw away the cached version? As Phil Karlton once said:
“There are two hard problems in computer science: cache invalidation and naming things”
The database may have been updated but the cached version contains old information. It is also difficult to provide personalised content for each user if you rely on a snapshot version of your page.
If things start to slow down, you can always increase the specification of your server right? If you are sure that this will solve the problem and you have the money to spend. But was the issue processing power, memory, disk based, network issues? Fixing any one of these could create bottlenecks and push the problem elsewhere. More powerful servers will inevitably increase your running costs.
This approach separates the responsibilities for each layer of your application. A typical 3 tier architecture has a presentation layer, a layer for business logic and a data layer. If a performance problem occurs at any one of these tiers, it can scale independently to the others. To do this each tier must run on separate servers. This introduces its own complexities of network communication latency and serialization/de-serialization of each request through the tiers. These applications are more complex, take longer to write and require more hardware than running the whole application together. Should your performance problems lie in your data tier, traditional relational databases are notoriously difficult to scale beyond a single server.
So What’s The Answer?
In the 1970’s, Hewitt introduced the Actor Model. All items in the application are created equal as an actor. One can communicate with another only by passing messages. Each message is processed one at a time in the order it was received. Until relatively recently, computers could not easily process multiple actors simultaneously. Their CPUs were single threaded and could execute one instruction at a time. Now even consumer CPUs are being produced with 16 cores with 32 available threads to execute instructions. Scaling the Actor Model involves simply adding more servers. Now is the time of the Actor Model! Several implementations of the Actor Model are now available such as Akka, MS Orleans and others written in Erlang and Scala.
This is not a one tool to replace all others. Sometimes your existing architecture is good enough. The Actor Model is complex, involves a steep learning curve and should only be considered where upwards of hundreds of thousands of transactions per second are required.
In a future post we will show you how we use the Actor Model, one way message passing and distributed computing to produce real-time updates across a multitude of devices.