Even within floating-point types alone, there are many different formats. You have probably heard of FP32, BF16, and FP64. They all follow the same three-part structure (sign, exponent, and mantissa), but they differ primarily in how many bits they allocate to the exponent versus the mantissa to balance range and precision. More bits for the exponent give a wider range of representable numbers, while more bits for the mantissa provide higher precision.

Formats are often described using the shorthand ExMy, where x is the number of exponent bits and y is the number of mantissa bits. For example, E4M3 uses 4 bits for the exponent and 3 bits for the mantissa, with 1 sign bit giving 8 bits in total, as shown in the figure below.

Similarly, BF16 is E8M7. FP16, which is also 16 bits total, is E5M10. It trades exponent range for mantissa precision, making it more likely to overflow at large values and to underflow to zero at small values. The table below shows the bit layout for the most common floating-point formats in ML.

While neural network weights and activations are overwhelmingly represented as floating-point numbers, machine learning systems also rely on a few other data types:

• Integers: Integers represent whole numbers with no fractional component, such as -1, 0, or 42.
• Booleans: Booleans represent binary truth values: either True or False.
• Complex numbers: Complex numbers have the form a + bi, where a and b are floating-point numbers representing the real and imaginary parts, and i = .

The table below summarizes these data types along with their bit widths and representable ranges.

You may be curious how 4-bit formats work because 4 bits is not even a full byte. Memory is addressed byte by byte, so numbers like INT4 and FP4 are packed two per byte. Some recent hardware can compute on 4-bit numbers directly. Otherwise, they are unpacked into a wider format before computation.