Number formats
How many bits a chip uses to store each number, from 64-bit floating point for science down to 8- and 4-bit formats for AI.
Also: FP64, FP32, FP16, BF16, FP8, FP4, INT8, precision
More bits give more accuracy but cost more silicon, memory and energy. Scientific simulation uses 64-bit (FP64) or 32-bit (FP32) floating point, following the IEEE 754 standard of 1985. Neural networks tolerate much less precision, so AI chips added 16-bit FP16 and Google's bfloat16 (BF16), then 8-bit FP8, adopted by Nvidia's Hopper in 2022, and 4-bit formats in Blackwell. Integer formats such as INT8 are used for inference. Halving the bits roughly doubles the throughput of the same hardware, which is why AI speed figures climb so steeply.