System Design Basics for Beginners: Scalability, Caching & Load Balancing

When an application grows from 100 users to 10,000,000 users, a single server can no longer handle the traffic. System design is the art of structuring distributed components to ensure reliability, high availability, and low latency.

---

1. Vertical Scaling vs Horizontal Scaling

• Vertical Scaling (Scale Up): Adding more CPU, RAM, and SSD storage to a single machine.
  • Limitation: Hard hardware ceiling, single point of failure (SPOF).
• Horizontal Scaling (Scale Out): Adding more commodity servers to distribute traffic across a cluster.
  • Advantage: High availability, virtually unlimited scalability.

---

2. Load Balancers (Traffic Distribution)

A Load Balancer acts as a reverse proxy, distributing client requests across multiple backend application servers.

Common Algorithms:

• Round Robin: Distributes requests sequentially across servers.
• Least Connections: Sends requests to the server with the fewest active sessions.
• Consistent Hashing: Used in distributed caching to minimize key re-mapping when nodes are added or removed.

---

3. Caching Strategies (Redis / Memcached)

Reading from disk or querying a database takes 5–50ms, while reading from an in-memory cache takes under 1ms.

Caching Invalidation Policies:

• Write-Through: Data is written to the cache and the database simultaneously.
• Write-Back (Write-Behind): Data is written to cache first and asynchronously flushed to database.
• Cache-Aside (Lazy Loading): Application checks cache first; on a cache miss, it reads from database and writes back to cache.

---

4. The CAP Theorem Explained

In a distributed system, you can only guarantee two out of three properties:

1. Consistency (C): Every read receives the most recent write or an error.
2. Availability (A): Every request receives a non-error response (without guarantee it contains the latest write).
3. Partition Tolerance (P): The system continues to operate despite network partitions or dropped packets.

Since network partitions are inevitable in real-world clouds, distributed systems are categorized as CP (e.g., MongoDB, HBase) or AP (e.g., Cassandra, DynamoDB).