In previous post, we discussed why cache is critical in computer systems. But cache is not golden, and it has certain limits due to conflicts. There are total of 4 limitations or 4C:
“Compulsory” happens during system boot time. Coming out of reset, cache is empty, and any cache access will result in a cache miss.
“Capacity” happens since cache is limited in size. Ideally, if the cache is infinitely large, everything is stored in cache, and there will be no cache misses. However, in real system, when cache is full, an access to a new cache block will result in a cache miss.
“Conflict” happens since different cache blocks may share the same cache location. An “later” cache block may evict an “earlier” cache block.
“Coherence” happens in cache-coherent system. For example, CPU A and B are in a cache-coherent system, and a cache block X is stored in CPU A’s cache. If CPU B needs a write to X, cache block X will be evicted from CPU A’s cache. Some time later, if CPU A needs to access X again, it will result in a cache miss.
By increasing cache size, “capacity” and “conflict” can be reduced, but “compulsory” and “coherence” remain the same.
By increasing cache associativity, “conflict” can be reduced, but other “3C” will remain the same.
In next post, we will discuss a famous interview question: “why is there no possible performance improvement with cache upsizing”.