Accumulator width, rounding and saturation

Accumulation needs its own width

Eight-bit operands and an eight-bit output do not imply an eight-bit accumulator. Signed eight-bit codes have maximum magnitude 128. With zero points zero and L terms, a conservative sum-magnitude bound is L×128×128, plus the magnitude of the bias. Choose a signed width whose positive maximum covers that bound; the sign bit consumes one bit.

Worked width calculation

For C=3 and a 3×3 kernel, L=27. When both operands are −128, each product is +16384 and the sum is 442368. Nineteen signed bits reach only 262143. Twenty bits reach 524287 and also cover a bias-magnitude allowance of 1000, giving bound 443368. Smaller proven operand ranges may reduce the required width, but a few ordinary test inputs cannot prove a bound.

Unsigned codes with zero point zero can reach 255, so the 128 assumption fails. First calculate A=max(abs(qmin−zx),abs(qmax−zx)) and the analogous B for weights. Use L×A×B+abs(b). Centering may itself require more bits than the stored code.

Fix the output-conversion order

The multiplier is sx×sw/sy. With multiplier 0.5 and output zero point 128, accumulator nine becomes tie 4.5, rounds to even four and produces 132. Accumulator eleven rounds 5.5 to six and produces 134. Accumulator −3 rounds −1.5 to −2. Truncation, truncation toward zero and arithmetic right shift differ for negative values. The exercises use a dedicated nearest-even routine rather than the default language rounding function.

For int8 output, converted 300 saturates to 127 and −300 to −128. Dropping high bits would wrap to unrelated values. Log values before clipping and the number of saturations. Include the location of ReLU and its representation of zero in the contract.

Verify large positive and negative sums, exact ties, both saturation boundaries and large bias. Deliberately contrast a wide reference with an undersized accumulator to expose overflow. Our runnable fixtures remain within exact JavaScript integer range; a larger reference must also justify the precision of its host arithmetic.