← Back to Field Notes

What Scalable Actually Means

architecture cloud engineering systems

Introduction

“Scalable” is one of the most overused words in engineering.

You’ll hear it everywhere:

  • “We need a scalable architecture.”
  • “This platform is designed to scale.”
  • “We built it this way so it’s scalable.”

But when you ask what that actually means, the answers often get vague.

Scalability doesn’t mean “uses Kubernetes”. It doesn’t mean “runs in the cloud”. And it definitely doesn’t mean “has a lot of microservices”.

Scalability simply means:

A system can handle increased load without breaking or becoming dramatically more expensive to operate.

But achieving that in practice requires more than just throwing infrastructure at a problem.

Vertical vs Horizontal Scaling

At the simplest level, there are two ways to scale a system.

Vertical Scaling

Vertical scaling means making a machine bigger - more CPU, memory, and storage.

Examples:

  • Upgrading a database from 4 CPUs to 32 CPUs
  • Moving a VM from a small instance to a large one
  • Adding more RAM to handle caching

Vertical scaling is simple and often very effective early on, but it does have limits. Eventually, machines can’t get bigger. And even when they can, the cost grows quickly.

Horizontal Scaling

Horizontal scaling means adding more machines instead of making one machine bigger.

Examples:

  • Adding more application servers behind a load balancer
  • Increasing the number of workers processing jobs
  • Running more containers in a cluster

Instead of one system handling all requests, the workload is distributed across many. This is how systems scale to very large workloads.

However, horizontal scaling introduces new problems, including coordination, data consistency, state management, and load balancing.

Scaling out is powerful, but it requires systems to be designed for it.

The Real Bottleneck Is Almost Always Data

In most architectures, compute is easy to scale. Stateless services can often be replicated indefinitely. The real challenge is usually state.

Examples include:

  • databases
  • caches
  • queues
  • shared storage
  • session data

Once multiple application instances need access to shared state, scaling becomes more complicated.

This is why you’ll often hear principles like:

  • stateless application servers
  • externalized session storage
  • distributed caches

These patterns make horizontal scaling possible by reducing the amount of state tied to individual machines.

Scaling Is About Removing Bottlenecks

Another misconception is that scalability is a single architectural decision. In reality, scalability is usually about identifying and removing bottlenecks.

For example - A system might scale well until:

  • the database connection pool fills up
  • a third-party API becomes slow
  • a queue consumer can’t keep up
  • disk I/O becomes saturated
  • a single background job blocks everything

Every system has bottlenecks. The goal isn’t to eliminate them completely - it’s to move them far enough away that they don’t impact normal operation.

Scalability is often iterative. You fix one bottleneck, and another appears.

Operational Scalability Matters Too

There’s another dimension that engineers sometimes overlook. A system isn’t truly scalable if the humans operating it can’t keep up.

Examples of operational bottlenecks:

  • manual deployments
  • fragile runbooks
  • undocumented systems
  • dashboards that don’t show useful information
  • alerts that fire constantly

A system that can handle 10x traffic but requires engineers to babysit it constantly isn’t really scalable.

Good scalable systems are observable, predictable, easy to operate, and resilient to failure.

Scaling Cost Matters

Handling more load is only useful if the cost grows reasonably. A system that handles 2x traffic but costs 10x more to run is not scalable in any meaningful way.

Good scalable architectures aim for the following:

  • roughly linear cost growth
  • efficient resource utilization
  • predictable infrastructure scaling

Cloud infrastructure makes scaling easier technically, but it also makes it easy to accidentally build systems that become extremely expensive at scale.

Simplicity Scales Better Than Cleverness

One of the most important lessons in real systems is this:

Simple architectures usually scale better than complicated ones.

Complex systems often introduce hidden dependencies and coordination problems that make scaling harder.

For example:

  • a small number of well-designed services often scale better than dozens of tightly coupled microservices
  • stateless services scale better than stateful ones
  • queues often scale better than synchronous request chains

Many scalability problems are really complexity problems.

Final Thoughts

Scalability isn’t a feature you add later - it’s a property of how systems are designed. But it also isn’t something you have to solve perfectly from the beginning.

Most systems don’t need to scale to millions of users on day one. What matters is building systems that:

  • remove obvious bottlenecks
  • scale incrementally
  • remain observable and operable as they grow

In the end, scalable systems aren’t defined by the technologies they use. They’re defined by how gracefully they handle growth.