Bandwidth and useful utilization

Define interval and units

An N×N grid peaks at N² MACs per advance edge. Fill and drain reduce utilization for an isolated job. In our unstalled, full-tile, one-edge-forwarding model, useful work is N²K and available slots are N²(K+2N−2), giving compute-interval utilization K/(K+2N−2).

For N=4, K=16, the grid performs 256 MACs in twenty-two advance edges. Utilization is 256/352=8/11, approximately 72.73%. At an assumed 100MHz, peak capacity is 1.6 GMAC/s and isolated compute-interval throughput is about 1.164 GMAC/s. Counting multiply and add separately doubles the operation number. These are estimates at an assumed frequency, not measured synthesis results.

Calculate traffic at each boundary

Injecting A and B requires 2NK values: 128 bytes for eight-bit operands. PEs consume 512 operand values, showing fourfold reuse of each boundary input. The overlapping injection interval can require eight bytes per edge, or 0.8GB/s at 100MHz. Peak supply demand differs from average demand over fill and drain.

Assume sixteen results use thirty-two-bit storage slots, despite the twenty-bit accumulator width. Result writes add sixty-four bytes, so external traffic totals 192 bytes without cross-tile reuse, command overhead or alignment waste. Arithmetic intensity is 256/192=4/3 MAC per byte.

At 0.4GB/s, transfer alone needs at least 480ns; computation needs 220ns. Even perfect overlap cannot beat the longer interval, bounding throughput near 0.533 GMAC/s. This is an upper bound, not guaranteed performance. Without overlap, loading takes thirty-two edges, computation twenty-two and storing sixteen, totaling seventy edges and about 0.366 GMAC/s before initialization or other waits.

The padded 3×3 example on N=2 has twenty-seven useful MACs in eighty PE slots: 33.75% useful utilization. Counting padded-zero MACs instead gives 60%, a different measurement. Jouppi et al.'s TPU paper is a research reference for separating peak accelerator capacity from workload performance; our numerical examples derive from our own schedule.