Caching: A Guide to Optimizing System Performance
Caching
Caches can be anything that reduces time and avoids expensive operations like Network I/O, Disk I/O, or computation.
What Needs to Be Cached?
API calls to get the profile of a user.
Queries involving multiple table joins.
Reading particular files repeatedly.
Caches store data in a temporary storage location for faster access. They are essentially key-value pairs.
How Cache is Fetched
Suppose a user makes a call to a backend API:
The API first checks if the data is available in the cache.
If not, it retrieves the data from the database, stores it in the cache, and returns the response to the user.
Fetching data from the database is time-consuming and costly. By storing frequently needed data in the cache, we can access it faster. While caching is an expensive operation, it is justified for important and frequently accessed data. However, storing all data in the cache would become too expensive.
Common Tools for Caching
Redis
Memcached
Note: Cache is not restricted to RAM-based storage; it can be any storage that is closer and helps avoid expensive operations.
Caches are essentially glorified hash tables.
Examples of Caching
Google News: Most recent news and articles are cached as they are more likely to be accessed.
Auth Token: Authentication tokens are cached to avoid repeated requests to the database.
Twitter: Recent posts from followed accounts are cached for faster access.
Live Streaming: The last 10 minutes of a livestream are cached on CDN for quick access.
Types of Caching
Lazy Population
The most popular way to cache. The process:
On read, first check the cache.
If the data exists, return it.
If not, retrieve it from the database, store it in the cache, and then return it.
Example: Caching a blog. Fetching data from the database is expensive, so it is stored in Redis or similar systems when accessed.
Eager Population
Data is written to both the database and cache simultaneously.
Example: Live cricket scores. Thousands of people watch cricket scores; serving them from the cache saves time and avoids cache misses.
Here, data is proactively pushed to the cache, especially for content that will definitely be accessed, like viral posts on social media.
Scaling the Cache
It is similar to a database, so we will use the same techniques to scale the database.
You can refer this to get detailed understanding :https://amanog.hashnode.dev/scaling-database
Vertical Scaling
Make the cache bigger to handle more data or load.
Horizontal Scaling - Replica (Scaling Read)
Replicate some data across nodes to save read operations.
Horizontal Scaling - Scaling Write
Distribute or shard data across multiple nodes to handle write operations efficiently. Shards are mutually exclusive.
Caching Challenges
Caching has problems like serving stale data and invalidation.
Caching at Different Levels
Client-Side Caching
Store frequently accessed data on the client side, such as in browsers or mobile devices.
Examples:
Constant data like images, JavaScript, and CSS files.
Recently watched content on streaming platforms like Amazon Prime is cached locally for quick access. However, logging into another device does not retrieve the same state.
Invalidation: Done through time expiry.
Content Delivery Network (CDN)
A set of distributed servers used for caching. Requests from users are routed to the nearest CDN server for faster responses.
Example: An application with servers in India serving a user in the US would benefit from caching images on a US-based CDN server.
CDN Cache Population: Done lazily. If the data is unavailable, the CDN requests it from the origin server, caches it, and sends the response to the user.
Remote Cache (Redis)
A centralized cache commonly used by multiple API servers to store or access frequently used data.
Database Caching
Store the results of expensive computational tasks for repeated use.
Example: Instead of running SELECT COUNT(*) FROM posts WHERE user_id = 123 every time, store the total posts in a database column and update it occasionally.
Remember, every time a post is published, we also update the user table by setting totalpost = totalpost + 1.
Note:
Whatever system you are designing or observing, each component has some memory where you can cache data, such as a load balancer. Just because you can add more caching layers doesn't mean you should. The more layers you add, the more stale data you might introduce.