Tech Roundup With Experts — The Holy Grail of Web Scalability

Startups face extreme amounts of uncertainty and challenges when their web applications' user-base multiplies. They need to be flexible in their choices, such as choosing the right framework, database, and architecture besides building the right team. What’s more, unlike enterprises, startups have only limited time to scale. Even worse, if they are fortunate enough, they might need to scale their capacity tenfold in a few months.

To understand the nuts and bolts of web scalability, we spoke to some experts and asked for their tips on improving web scalability. Here’s what they said…

You may also like: Scalability Introduction for Software Engineers.

The Nuts and Bolts of Web Scalability

Choosing the Right Tool From the Beginning

Choosing a web framework with scalability in mind should be the first step in ensuring scalability. If a website is built with a strong framework from the start, it could perform well in other aspects of business (e.g. finance, marketing, management, HR, etc). 

Think about it like this: If you lay a solid foundation for your house (website) from the start, you can expect to build it, even more, stronger and larger in the future.

But don’t fall prey to the idea of sticking with the same framework. It’s impossible to scale a website built on PHP to billions of users. That being said, if the situation presents itself where you might turn the tables around and sunset a particular tech stack –– do it without hesitation.

In the words of Aditya Agarwal (VP of Engineering at the DropBox), “Realize with that choice there is overhead, usually across deployment, monitoring, ops, and so on. If you choose to use a services architecture, you'll have to build most of the backend yourself and that often takes quite a bit of time. With the LAMP stack, you get a lot for free. Once you move away for the LAMP stack, how do things like service configuration and monitoring is up to you. As you go deeper into the services approach you to have to reinvent the wheel."

Implement Multiple Levels of Caching to Reduce Chances of a Cache Miss

Have you ever noticed that when you load a web page for the very first time, it takes a little bit longer than the second, third, etc. times that you visit that page? That’s an intelligent logic that goes behind the scenes in a web browser –– caching.

Caching makes it easier for an application to perform as it avoids or delays requests to the server every time a page has to load. However, caching in the case of complex apps is tricky. Here’s why it is advisable to implement multiple levels of caching in an application to make it scalable.

According to Jared Rerie, “The argument that 'the server is caching, so why should the client?' is very common in my experience. Perhaps it’s because we’re taught as developers to avoid duplication (Don’t Repeat Yourself). One might argue that several teams implementing caching is a form of wasteful duplication of effort. While multiple layers of caching can introduce complexity to a system, it’s not wasteful and not even true duplication because client-side and server-side caching are potentially very different.”

Using Multiple Databases to Maintain Data Integrity in Complex Systems

It often becomes tedious to choose between a SQL and NoSQL database while developing an application that scales. Even worse, the marketing wave of both kinds of databases has made this decision even more difficult.

In the words of Frederic Faure, Architect at Ysance, “I find the positioning of SQL and NoSQL (Not Only SQL) as opposites rather a shame: it’s true that the marketing wave of NoSQL has enabled the renewed promotion of a system that’s been around for quite a while, but which was only rarely considered in most cases, as after all, everything could be fitted into the « good old SQL model ». The reverse trend of wanting to make everything fit the NoSQL model is not very profitable either.”

In fact, a survey done by scalegrid.io suggested that 44.3% of businesses amalgamate two or three databases to scale their application. What’s more, SQL and NoSQL was the most popular multi-database combination employed by over 75.6% users.

Here’s why using multiple databases has a significant impact on improving website scalability.

Don’t Store State Unless You Have a Good Reason

Stateful vs Stateless is a common debate all over the web with developers divided over the benefits of both.

What all these debates have in common has little to no significance with respect to scalability. Stateless, which relates to not storing any state in your web applications, allows you to simplify the client/server communication model of your application. The major benefit of this is that the client can request anything from the server without storing the state of the server. Moreover, since there is no need to change the pool of sessions throughout the web servers, your application can scale out without any hassle.

On the other hand, there may be some use-case when it is a good idea to store state in your web application. Hughe Mckee, the developer advocate at Lightbend, explains the benefit of preserving state in a shopping cart application in his blog “How to build successful cloud-native solutions”. The article illustrates the scaling capability of stateful application with a shopping cart example.

Per Hughe, “When it comes to cloud-native software systems, determining the best approach depends on the specific circumstances. This certainly applies when considering stateless versus stateful systems. In many cases, the stateless approach is an acceptable solution; however, there is a growing number of scenarios where using a stateful approach will be a better alternative. This is undoubtedly true for the ever-increasing demand for high-performance near real-time and stream-based systems. “

Go Asynchronous

It goes without saying that “Asynchronous programming is the proven way of increasing web scalability.”

But how true is this claim? Udi Dahan might have the answer.

In the words of Udi Dahan, “Often, during my consulting engagements, I run into people who say, 'some things just can’t be made asynchronous' even after they agree about the inherent scalability that the asynchronous communication pattern brings. One often-cited example is user authentication — taking a username and password combo and authenticating it against some back-end store."

When you are using Asynchronous IO, you are handling thousands of requests at the same time without blocking any of them. Some single-threaded frameworks, such as Node.js, do this efficiently and more effectively. 

Avoid Any Single Point of Failure

Failures are inevitable. Web applications need to be immune to any single point of failure. Whether it be your server, database, message queue, disk-drives –– all must be isolated and independent of functioning on their own to ensure that when they fail, your entire system doesn’t break down.

But, before you decide to isolate any failure or root-out any SPOF, ask yourself a certain set of questions: What if the piece of code you have written doesn’t work at all? What if you end up writing an old legacy code that is hard to maintain and scale at some point?

Sean Hull, Cofounder of Orbit, advises you to root out any single point of failure before preparing your system to scale.

In the words of Sean, “If your data is on a single master database, that's a single point of failure. If your server is sitting on a single disk, that's a single point of failure. This is just technical vernacular for an Achilles heel. These single points of failure must be rooted out at all costs.”

Further Reading

• Asynchronous Programming.
• Architecting App Logic for Scalability, Reuse, and Agility.
• Serverless And Startups: Remodeling The Startups Landscape.