Chapter 7.
Performance metrics

The previous chapter gave us the mental model: an op is either memory-bound or compute-bound (concepts introduced in Memory-bound and Compute-bound), and pipelining and op fusion are the tools to address the former. But those labels are qualitative. To actually diagnose an op, or predict its performance before writing any code, we need quantitative tools. In this chapter, we introduce throughput to measure how fast hardware and ops run, arithmetic intensity to predict which side of the bottleneck an op falls on, and the roofline model that ties the two together into a single performance ceiling.