Interviewers often ask how to design a memory controller in technical interviews. We show one example below.

The memory controller takes incoming requests along with address and request ID as inputs. It is expected to provide read responses along with response ID as outputs. Internally, it can access memory to fetch the read data.

The memory controller needs to meet the following 2 requirements:

    1. The order of data responses must follow the incoming request, i.e., RSP_ID must follow the same sequence as REQ_ID
    2. Data throughput cannot be compromised, i.e., we need to sustain requests every single cycle

There will be 2 cases to consider:

    1. The memory access latency is a constant, e.g., always 10 cycles
    2. The memory access latency is a variable, e.g., up to 10 cycles

Constant Memory Access Latency

If the memory access latency is a constant, meeting the 1st requirement is straightforward. We need to keep a FIFO inside the memory controller to associate the RSP_ID with response data. 

Note, no read to memory can be issued if FIFO has no spaces. Upon issuing a read to memory, we store the REQ_ID into the FIFO; upon the receipt of read data from memory, FIFO output will drive RSP_ID.

To meet the 2nd requirement, the FIFO depth has to be at least 11. 10 cycles later when the data response of the 1st request comes back, FIFO should still have space to store REQ_ID so that the 11th request can be issued.

Obviously, the FIFO is not needed if downstream logic does not need to associate the RSP_ID with response data.

Variable Memory Access Latency

For memories like DRAM, keeping memory access latency constant is unrealistic since it introduces severe performance degradation. Thus variable memory access latency is more common in the real world.

To meet the 1st requirement, we need to keep a reorder buffer or ROB to associate the RSP_ID with response data. Again, no read to memory can be issued if ROB has no spaces.

Each ROB entry has 3 fields: valid bit, read data, and REQ_ID. Entries are pushed into or popped from ROB in order.

Upon the receipt of a new request, we store REQ_ID in tail entry of the ROB; upon the receipt of read data from memory, we set the valid bit and store read data into corresponding ROB entry. Note, memory controller may not provide data responses to downstream logic at this point, and ROB may not be popped.

Only when the head entry of ROB has valid bit set, should we pop ROB and provide data responses to downstream logic.

To avoid the throughput loss and meet the 2nd requirement, ROB depth has to be at least 11.

Apparently, ROB is always needed, no matter downstream logic needs to associate the RSP_ID with response data or not.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.