Distinguishing cycle latency from throughput

Define where “N cycles later” starts

A bit first captured in bit 0 at edge k reaches bit N-1 of an N-bit shift register^{iRegister A circuit that stores multiple bits of state. The synchronous registers in this course store their specified inputs at a clock edge. Learn more} at edge k+N-1. The interval from first capture to final capture is N-1 periods.

If instead an upstream register launches the new bit at edge k, the first shift-register capture normally occurs at k+1. The output is then N periods from launch. These statements have different starting points; they do not contradict each other.

Define d[k] as the input accepted at edge k. Once the register is filled,

Long latency can still allow a result every cycle

With clock period T and one accepted bit per edge, steady-state throughput^{iThroughput The number of operations accepted or completed per unit time. A pipeline can increase throughput without reducing the latency of an individual operation.} is $1/T$ bits/second. Increasing N increases storage depth and latency, but throughput stays the same as long as input can be accepted continuously.

A four-stage delay line at 100 MHz has a 10 ns period and steady-state throughput of 100 Mbit/s. “One bit takes time to traverse the line” differs from “you must wait that long before inserting the next bit.”

Each column below records a rising edge^{iRising edge The instant at which the clock changes from 0 to 1. Distinguish it from a level, which refers to the entire interval during which CLK=1. Learn more}. Inputs are pre-edge; states labeled “after” are post-update. Alignment shows sampling order, not physical propagation delay^{iPropagation delay The time from an input change until the output settles to the correct value. Logical equivalence and timing behavior are separate properties. Learn more}. In the three-stage line, A enters stage 0 at E0 and reaches the last stage just after E2, while B and C occupy earlier stages. Dashes mark slots not yet valid.

With stalls, count accepted shifts

An edge with enable or accept=0 does not shift bits. There is then no fixed elapsed-time latency, only a latency in accepted edges. Shift valid under the same conditions as data to identify when the output is meaningful.

Describing a pipeline^{iPipeline A structure that divides computation into register-separated stages and overlaps processing of multiple inputs. Data, valid and control signals must be aligned by stage. Learn more} only as a latency-reduction technique is misleading. It often divides combinational paths to increase clock frequency while adding register stages and cycle latency. Area, power and frequency tradeoffs are explored in more detail in the following course.