These are the levels in a typical memory hierarchy. Moving farther away from the CPU, the memory in the level becomes larger and slower .

The above principles suggest that we should try to keep recently accessed items in the fastest memory. Because the smaller memories are more expensive and faster, we want to use smaller memories to try to hold the most recently accessed items close to the CPU and successively larger (and slower, and less expensive) memories as we move away from the CPU. This type of organization is called a memory hierarchy .  Two important levels of the memory hierarchy are the cache and virtual memory.

To evaluate the effectiveness of the memory hierarchy we can use the formula:

      Memory_stall_cycles =  IC * Mem_Refs * Miss_Rate * Miss_Penalty

         where       IC                       = Instruction count
                        Mem_Refs           = Memory References per Instruction
                        Miss_Rate            =  the fraction of accesses that are not in the cache
                        Miss_Penalty       =  the additional time to service the miss