What is INT4? Four-bit storage does not define all arithmetic
Explain INT4 quantization, distinguish other four-bit formats and check scales, metadata and backend compatibility.
Understand the concept
INT4 is four-bit integer representation used in low-bit quantization. It differs from formats such as FP4, and INT4 weights do not mean every activation and operation uses four bits. Check schemes, backends and quality. Metadata and runtime data prevent exact size predictions from bit ratios.
Quantized values need mapping to the original range
TensorRT distinguishes INT4 and FP4 with specific support, while GPTQ approximates weights after training. Scales and other metadata map integers to original floating values. Grouping and quantized components affect behavior. A four-bit label needs an actual encoding; methods are not interchangeable formats. Loading compatibility is only the start of quality validation.
Check four-bit outputs against the same base model
Use comparable forms of one model on fixed public identifiers and amounts and retain raw errors. Record method, framework and device rather than only INT4. Check format support on loading failure. No model was quantized or run here. Test long contexts and concurrency separately; weight compression does not imply equal cache compression.
Preserve encoding and recoverable deployment settings
Replay successes and failures after conversion or backend updates. Keep usable originals and conversion details instead of an untraceable compressed file. Include conversion, storage and correction effort. Consider other precision or task-specific handling when important tasks degrade. INT4 addresses representation without guaranteeing speed or replacing capability checks.
Checked sources
These sources support the concepts and methods described here. Examples and test scope are labeled separately; check the version of the tools you use.