Why so many floats? Model data types, bit by bit

When you download a model from Hugging Face, the same model often comes in several versions. Qwen3-8B¹, for example, comes as a BF16 version and an FP8 version; other models are released in FP32², such as OLMo-2 7B. The difference is how many bytes each parameter uses: 4 for FP32, 2 for BF16, 1 for FP8. A 7-billion-parameter model takes 28 GB in FP32, 14 GB in BF16 and only 7 GB in FP8³. What exactly is lost with the missing bytes? Let’s start from the bits of one number.

How a float is stored

A floating-point number has three parts

A floating-point number is scientific notation in binary. In decimal we write 0.0123 as 1.23 × 10⁻²; the computer uses base 2 instead: value = ±1.mantissa × 2^exponent, for example 0.0123 = +1.5744 × 2⁻⁷. So it only has to store three things:

• Sign: 1 bit, positive or negative.
• Exponent: sets the magnitude, the power of 2 the number is multiplied by. More exponent bits cover a wider range, from very small to very large numbers.
• Mantissa: sets the precision. More mantissa bits mean a more exact number.

The formats differ only in how many bits they give to exponent and mantissa:

FP16 and BF16 both use 2 bytes but split them differently: FP16 favours precision, BF16 favours range.

In the short name, E is the number of exponent bits and M the number of mantissa bits; add the 1 sign bit and you get the total. FP8 and FP4 are usually named this way to tell apart different splits of the same size.

One number, stored different ways

Take the number 0.0123 (a typical size for a model weight). Each box is one bit: red is the sign, blue the exponent, green the mantissa; the value after the arrow is what actually gets stored:

Three things stand out:

• BF16 has exactly the same exponent as FP32 (01111000 in both); the mantissa keeps only the first 7 bits, rounded. That is how BF16 was designed: it is the first 16 bits of FP32.
• FP8 stores 0.0117, an error of almost 5 %. 0.0123 is already below 0.0156, the smallest normal value of E4M3, so the exponent is all zeros and the 3 mantissa bits have to do all the work. E5M2 has one more exponent bit, so 0.0123 is in its normal range, but with only 2 mantissa bits it also ends up at 0.0117.
• FP4 is even worse: 0.0123 becomes 0. That is why FP8 and FP4 must be scaled first, as explained below.

The common formats

Why training uses BF16, not FP16

FP16 has only 5 exponent bits: its largest value is 65504 and its smallest normal value 0.00006. Gradients during training are often smaller than that and turn into 0 when stored as FP16. Training in FP16 therefore needs loss scaling: multiply the loss by a large number to lift the gradients into FP16’s range, then divide again before the update. This is cumbersome.

BF16 has 8 exponent bits like FP32, so the range is exactly the same: no overflow, and underflow is rare. What it gives up is precision, only 2 to 3 significant digits, and neural networks barely notice that error. Since the A100, large models have been trained and released in BF16 almost everywhere.

FP8: scale first, then store

FP8 has only 1 byte; the common E4M3 format (4 exponent bits, 3 mantissa bits) goes up to just 448. As the example above shows, converting weights directly to FP8 pushes many values below its smallest normal number and loses a lot of precision.

So FP8 models scale first. Qwen3-FP8 and DeepSeek-V3, for example, cut each weight matrix into 128 × 128 tiles (weight_block_size in the config), and the 16,384 values of a tile share one FP32 scale factor, which stretches the block’s largest absolute value to about 448 before conversion. Each block then uses FP8’s full range. The scale factors are visible in the safetensors files, usually under the name weight_scale_inv.

The other FP8 format, E5M2 (5 exponent bits, 2 mantissa bits), has a larger range and lower precision and is mostly used for gradients.

Further down: FP4

FP4 has only 4 bits. The common E2M1 format has 1 sign bit, 2 exponent bits and 1 mantissa bit, 16 codes in total; the positive values it can represent are just 0, 0.5, 1, 1.5, 2, 3, 4 and 6. A format this coarse cannot be used on its own and always comes with scaling: MXFP4 shares one scale factor among every 32 numbers, NVIDIA’s NVFP4 shares one FP8 scale factor among every 16 numbers; the smaller the block, the closer the scaling fits the data. FP4 is used mainly for inference today, and the Blackwell generation of GPUs is the first to support it directly in hardware.

Block-scaled formats: MXFP8, MXFP4, NVFP4

“One scale factor per block of numbers” has since become a set of standard formats that GPUs support in hardware. The common ones side by side:

The scale factors take space too: NVFP4 adds one 8-bit scale per 16 numbers, half a bit per number. NVFP4 spends 0.25 bits more than MXFP4 and gets two things for it: blocks half the size, so the scale fits the data more closely, and an E4M3 scale instead of a bare power of 2, so the scaling itself is more exact.

Compression

The exponent compresses, the mantissa does not

Model weights are mostly small numbers close to 0, and their exponents cluster on a few values. So the truly “random” part of a number is the mantissa, while the exponent is easy to predict. The figure below splits all BF16 weights of Qwen3-1.7B into two parts: on the left the sign plus the 8 exponent bits, 9 bits with 512 possible values; on the right the 7 mantissa bits, with 128 possible values. It shows how often each value occurs:

The left side sits almost entirely on two narrow clusters, one for positive weights and one for negative ones; they have the same shape and differ only in the sign bit. Of the 512 possible values, 18 hold 99 % of the weights, so this part compresses well. The mantissa on the right is spread flat, every value at about 0.8 %, and barely compresses.

Entropy measures this: the average information a group of bits really carries, at most its number of bits; when the two are equal, the bits are completely random and do not compress. The figure shows the entropy of the exponent and of the mantissa for each format, as a share of their bits:

The mantissa is close to 100 % in every format, almost random. The exponent is different: the 8 exponent bits of FP32 and BF16 carry only 2.6 to 2.8 bits of information, two thirds of them are redundant, and that is the part that can be compressed. FP16 has only 5 exponent bits and uses half of them. After block scaling, FP8 uses 2.7 of its 4 exponent bits; FP4 fills its 2 exponent bits completely. The next section works out how far each format can shrink.

How far can they be compressed

Sign and mantissa are close to random and do not compress; only the exponent does. That gives two bounds:

• Limit: assume the exponent costs nothing and store only sign and mantissa. Ratio = total bits ÷ (1 + mantissa bits).
• Realistic bound: store the exponent at its entropy in real weights, sign and mantissa as they are. This is the best any coder can do that looks at each value on its own, without context.

Take BF16 and Qwen3-1.7B: a number has 16 bits; the sign bit and the 7 mantissa bits are stored as they are, 8 bits in total. The 8 exponent bits could hold 256 values, but real weights use only a dozen of them, and very unevenly; with entropy coding (short codes for frequent values, long codes for rare ones) an exponent takes 2.6 bits on average. A compressed number then takes 1 + 2.6 + 7 = 10.6 bits on average, so the realistic bound is 16 ÷ 10.6 ≈ 1.51. The limit assumes the exponent costs nothing: 16 ÷ (1 + 7) = 2.00.

Why not compress the sign? Weights are half positive and half negative, so the sign bit has an entropy of exactly 1 bit and there is nothing to save. Coding sign and exponent together does not help either: positive and negative weights have almost the same exponent distribution, so the sign tells nothing about the exponent, and coding them together gives the same result as coding them apart.

The figure shows three well-known models per format⁶. Each panel title gives the bits of sign, exponent and mantissa; the bars are the realistic bound, with their formula in blue, where E is the number of bits the model’s exponent really needs on average (its entropy), written under the model name; the grey dashed line is the limit, labelled with its own formula. The ratio is original size divided by compressed size; 2.00 means half the size. Models of the same format differ very little; all three BF16 models, for example, reach 1.51.

BF16 gains the most: its 8 exponent bits take half the space but carry only 2.6 bits of information in real weights, so it can shrink to 66 % of its size, a ratio of 1.51. FP32’s exponent also carries only 2.8 bits, but the 23 mantissa bits dominate and cap it at 1.19. FP16 has only 5 exponent bits to begin with, so there is little to save.

The smaller the format, the less it compresses. After block scaling, FP8 uses 2.7 of its 4 exponent bits and is left with 1.20; FP4 fills its 2 exponent bits almost completely and has essentially no room. Scaling exists to use every bit, and data that uses every bit leaves lossless compression nothing to save.