What is FP16? Half-precision capacity and numerical limits
Explain sixteen-bit floating point, distinguish BF16 and mixed precision and verify labels against actual runtime.
Understand the concept
FP16 usually means sixteen-bit half-precision floating point, with different range and precision from FP32. BF16 is another layout. An FP16 label does not describe total service memory or guarantee speed over every format. Check hardware, arithmetic and task results.
Equal bit widths do not imply equal numerical ranges
NVIDIA describes half-precision sign, exponent and significand, and CUDA defines related operations. Approximation has overflow and underflow limits. Mixed precision uses multiple formats and retains higher precision where needed. Training and inference have different resource profiles. Inspect actual weights and backend settings rather than filenames.
Check precision changes on the same model
Compare fields from one model before and after conversion on public numerical inputs, then test long contexts. Separate invalid values from backend incompatibility and retain errors. Smaller files do not prove equal quality and hardware-document results are not your tests. No conversion or device run occurred here. Fix versions and task conditions for interpretable comparisons.
Account for runtime data when estimating savings
Weights are only part of total memory; caches and temporaries may use other formats. Unsupported execution paths can prevent latency gains. Replay compatibility and quality after upgrades and retain recovery versions. Include conversion and review, not bit-ratio cost estimates. Half precision implies neither half the capability nor universally lossless operation.
Checked sources
These sources support the concepts and methods described here. Examples and test scope are labeled separately; check the version of the tools you use.