im2col and matrix multiplication

Re-express the operation

im2col flattens each input window into a vector for matrix multiplication. We place one patch in each row. Matrix A has P×K elements, with P=HoWo and K=CRS. Flattened weights form columns of matrix B, shape K×M. The product AB has shape P×M. Other conventions place patches in columns, so establish the axes rather than inferring them from the name.

You may materialize the full patch matrix or generate patches on demand. Both can produce the same numbers with different storage and bandwidth. Having a matrix-multiplication engine does not require writing a complete expanded matrix to external memory.

Worked matrix

For [1,2,3;4,5,6;7,8,9] and 2×2 windows, A has rows [1,2,4,5], [2,3,5,6], [4,5,7,8] and [5,6,8,9]. Kernel [1,2;3,4] gives weight column [1,2,3,4]. The four dot products are 37,47,67,77. Thus 4×4 multiplied by 4×1 yields 4×1, which folds back into a 2×2 output.

The original input contains nine elements; the expanded matrix contains sixteen. Center value five appears four times, while corner value one appears once. Spatial reuse becomes duplication in a materialized representation. One weight vector is reused for all four patches. Calculate P×K and compare it with C×H×W before assigning storage. A larger output region can make duplication substantial.

For multiple channels, flatten the patch and weights in the same c,r,s order. A two-channel 1×1 patch is [x0,x1]; reversing only the weight channels changes the result. Start with asymmetric channel values. Grouped convolution forms separate group vectors rather than indiscriminately including every input channel.

Agreement with a direct model verifies arithmetic values, not physical memory transfers or simultaneous PE operation. The next chapter generates equivalent windows from a stream; later tiling avoids storing a whole expanded tensor. Record intermediate matrix size and duplication as well as output values so that mapping costs remain visible.