Connecting ReLU and max pooling

Include the post-processing stages

Inference may continue beyond convolution. ReLU selects the larger of each real value and zero. Max pooling selects the largest value in a spatial window. One is elementwise; the other changes spatial size and needs a different completion condition. Our path requantizes, applies ReLU, then performs unpadded 2×2 max pooling with stride two. The floor convention discards an incomplete final row or column.

For raw quantized output, real zero is output zero point zy. A positive scale preserves ordering, so ReLU uses max(q,zy). Comparing uint8 codes with literal zero when zy=128 would preserve negative real values incorrectly.

Worked chain

Accumulators [−2,4;6,−8] with multiplier 0.5 and zero point zero become [−1,2;3,−4]. ReLU produces [0,2;3,0], and pooling selects three. Spatial downsampling and numerical rescaling are separate ideas. Max pooling picks an existing code and therefore preserves its scale in this exercise.

With zero point 128, codes [126,132;134,120] become [128,132;134,128] after ReLU and pool to code 134. The reference remains real zero even when the storage representation changes. Average pooling would introduce division and rounding, so it is not interchangeable with this implementation.

A negative pre-ReLU window [−8,−5;−3,−6] has maximum −3. Incorrectly inserting zero at a padded boundary lets a nonexistent sample win. A padded implementation must exclude that location or use the specified minimum value. Our runnable pooling function supports no padding.

Fusing stages may reduce intermediate memory traffic, but preserve observable values for verification. Compare accumulators, requantized codes, activated values and pooled output separately. A 4×4 input yields four pooled values; a 3×3 input yields one under the stated floor rule. Document incomplete-window policy in tests. See the primary definitions in ONNX Relu and ONNX MaxPool.