Bandwidth and compute bounds

Feed the arithmetic

MAC count measures work; bandwidth measures how quickly operands can reach it. Four single-MAC PEs offer an ideal four MACs per cycle. Fetching two one-byte operands for every MAC would demand eight bytes per cycle. Reuse lowers external traffic, but local buffers still need enough ports to feed the PEs. External bandwidth and internal read-port demand are different quantities.

The compute lower bound is MAC count divided by available MAC rate. The transfer lower bound is bytes divided by bandwidth. With perfect overlap, total time cannot beat their maximum. Without overlap, add them. Startup, bursts, address setup, tails and writing can increase an actual schedule.

Worked hypothetical design

Use N=1, C=2, H=W=6, M=4 and valid 3×3 convolution. Output is 4×4 and work is 1152 MACs. Suppose every tensor is fetched once, with no halo reloads or partial-sum spills. Int8 inputs use 72 bytes, weights 72, outputs 64 and 32-bit biases 16: 224 total bytes.

Four PEs give a compute bound of 288 cycles. At one byte per cycle, transfer requires 224 cycles, so the perfect-overlap bound is 288. At half a byte per cycle, transfers require 448 cycles and become the larger bound. Without overlap at one byte per cycle, the simple sum is 512 cycles. Attach these assumptions to every reported number.

Under this reuse assumption, arithmetic intensity is 1152/224≈5.14 MACs per byte. Fetching both operands for every MAC instead uses 2304 operand bytes; outputs and biases raise the total to 2384. The work is unchanged; movement differs. This intentionally ignores caches, compression and bus alignment and is not a hardware measurement.

Record unique tensor size, actual read/write bytes, partial-sum traffic, useful PE operations and total cycles. Classify idle cycles by input starvation, output blocking, tails and control. Doubling PEs cannot halve time when transfers dominate. Better reuse may need more storage, so consider this analysis together with the previous tile budget.