Scale and zero point

The code for real zero

Affine quantization uses a positive scale s and integer zero point z. Our integer contract rounds x/s to the nearest integer, selects the even integer at an exact tie, adds z, then saturates to the storage range. Dequantization interprets a code as approximately s(q−z). Mapping real values onto an integer grid generally loses information.

For uint8 input with s=0.25,z=128, real zero is code 128. Codes 130 and 126 represent 0.5 and −0.5. Raw padding code zero would instead represent −32. A buffer storing raw codes must insert z for padding. A buffer storing already centered codes inserts integer zero. Mark that conversion point in the datapath.

Worked dot product

Center input codes [130,126] by subtracting 128, obtaining [2,−2]. Weights [3,−2] use scale 0.5 and zero point zero. Accumulate 2×3+(−2)×(−2)=10. The accumulator's real scale is 0.25×0.5=0.125, so the real result is 1.25. With output scale 0.25 and zero point 128, it becomes code 133. Nonzero weight zero points must also be subtracted before multiplying.

Per-tensor quantization shares a scale. Per-channel weight quantization requires channel-specific bias and output conversion. If two output channels have weight scales 0.5 and 0.25, accumulator code ten represents 1.25 and 0.625 respectively. Software flexibility does not establish hardware support for either arrangement.

Quantizing 0.625 at scale 0.25 gives the tie 2.5, which rounds to two and code 130. Dequantization gives 0.5, an error of −0.125. Input 0.375 also maps to that code. Quantization-grid error differs from saturation error: clipping an out-of-range input can exceed half a scale step. Compare the integer rules with ONNX QuantizeLinear. Our exercises deliberately implement only a small int8/uint8 subset.