https://github.com/G33kzD3n/Catalogue/blob/master/System Design Interview An Insider’s Guide by Alex Xu (z-lib.org).pdf

Chapter 1: Scaling from Zero to Millions of Users

Scaling a system is an iterative process. Iterating on what we have learned in this chapter could get us far. More fine-tuning and new strategies are needed to scale beyond millions of users. For example, you might need to optimize your system and decouple the system to even smaller services. All the techniques learned in this chapter should provide a good foundation to tackle new challenges. To conclude this chapter, we provide a summary of how we scale our system to support millions of users:

Keep web tier stateless

• The stateless architecture

Now it is time to consider scaling the web tier horizontally.

For this, we need to move state (for instance user session data) out of the web tier. A good practice is to store session data in the persistent storage such as relational database or NoSQL. Each web server in the cluster can access state data from databases.

Cache data as much as you can

Here are a few considerations for using a cache system:

• Decide when to use cache: when data is read frequently but modified infrequently.
• Expiration policy: When there is no expiration policy, cached data will be stored in the memory permanently. It is advisable not to make the expiration date too short as this will cause the system to reload data from the database too frequently. Meanwhile, it is advisable not to make the expiration date too long as the data can become stale.
• Consistency: This involves keeping the data store and the cache in sync.
• Mitigating failures: A single cache server represents a potential “A single point of failure (SPOF) is part of a system that, if it fails, will stop the entire system from working”
• Eviction Policy: Once the cache is full, any requests to add items to the cache might cause existing items to be removed. This is called cache eviction. Least-recently-used (LRU) is the most popular cache eviction policy. Other eviction policies, such as the Least Frequently Used (LFU) or First in First Out (FIFO), can be adopted to satisfy different use cases.

Support multiple data centers

Several technical challenges must be resolved to achieve multi-data center setup:

• Traffic redirection: Effective tools are needed to direct traffic to the correct data center. GeoDNS can be used to direct traffic to the nearest data center depending on where a user is located.
• Data synchronization: Users from different regions could use different local databases or caches. In failover cases, traffic might be routed to a data center where data is unavailable. A common strategy is to replicate data across multiple data centers.