4.1. Memory wall
When we parallelize a computation, we assume that the cores are the bottleneck. If they finish work faster, the whole program finishes faster. This assumption breaks down when the cores spend most of their time waiting for data to arrive from memory rather than actually computing. This "slow memory" problem is known as the memory wall.
A core's activity over time looks something like this.
Figure 8. A core's activity over time.
Most of the time, the core is not computing at all. It is just waiting for data. Making the compute faster would barely change the overall runtime.