Convolution and cross-correlation

Establish kernel orientation

Neural-network convolution commonly uses cross-correlation: pair the stored kernel with the input window without reversing it. Mathematical convolution reverses the kernel. The index equation is more useful than the operator name. A symmetric kernel can hide an orientation mistake, so begin verification with asymmetric values.

For one channel, unit stride and no padding or dilation, use:

Here h,w locate the output and r,s locate a kernel element. Neither kernel index is reversed.

Worked example

Use the input window [1,2;4,5] and kernel [1,2;3,4]. Cross-correlation yields 1×1+2×2+4×3+5×4=37. Reversing both kernel axes gives [4,3;2,1], producing 4+6+8+5=23. Adding a bias of two produces 39 and 25. The specification decides which operation is intended.

Extend the input to [1,2,3;4,5,6;7,8,9]. The original kernel produces [37,47;67,77]. The top-right window is [2,3;5,6], whose products sum to 2+6+15+24=47. Moving a window changes input coordinates; it never changes the order of the stored weights. Negative weights obey the same indexing rule. For example, [1,0,-1] explicitly subtracts the right sample from the left.

With multiple channels, accumulate each channel's window into the same output element. Each output channel has its own collection of kernels and one bias. Channels are reduction indices, rather than spatial positions along which a window slides. Grouped convolution restricts those connections and needs an explicit group mapping.

Log the input window, kernel order, partial products and bias. Checking only a final array can miss compensating indexing errors on overly regular fixtures. The asymmetric examples here reduce that risk. Compare tensor and weight conventions with the official ONNX Conv documentation; the numerical fixtures in this lesson are independently constructed.