The pipeline shown in the previous section is ideal. In reality, the read, compute, and write steps may take different amounts of time. The following figure shows an example where the read and write steps take twice as long as the compute step. The processors are no longer busy all the time but have a lot of idle time waiting for the read step to complete.

Copying data across the memory hierarchy becomes the performance bottleneck. Reducing the computation time will not help improve the overall performance much. However, if we can optimize the memory access, for example, reducing the data transfer between global and shared memory, we can significantly improve the overall performance. This is known as being memory-bound.