hop 22 of 32 · 5 min
Not overwhelming the receiver
A fast sender and a slow receiver is a problem the network cannot solve. Flow control is the receiver telling the sender how much it can take.
Stop and wait is correct and slow
Send one segment, wait for the acknowledgement, send the next. Correct, and it wastes the link: on a connection with 100ms round trip and 100 Mbps capacity, sending 1,500 bytes and waiting 100ms achieves about 120 kbps. Roughly a thousandth of the link.
Sliding window
Allow several segments in flight at once. The receiver advertises a window: how many bytes it can accept beyond what it has acknowledged.
sent & acked | sent, not acked | may send | cannot yet
-------------|=================|==========|-----------
^ ^
window start window endAs acknowledgements arrive the window slides forward and more may be sent. The window size is carried in every segment, so it adjusts continuously as the application drains the buffer faster or slower.
The number that decides throughput
maximum throughput = window size / round trip timeThis is the bandwidth delay product, and it explains a class of problem people usually misdiagnose as a slow network.
A 1 Gbps link with 100ms round trip needs a window of 12.5 MB to be filled. The original TCP window field is 16 bits, so it maxes at 65,535 bytes, which on that link gives about 5 Mbps regardless of the capacity available.
Window scaling multiplies the field by a negotiated power of two and removes the ceiling. It is negotiated in the handshake, and a middlebox that strips the option silently caps a fast long link at a few megabits, which is maddening to diagnose because everything reports as working.
| Round trip | Window 64 KB | Window 1 MB |
|---|---|---|
| 1 ms | 500 Mbps | more than the link |
| 50 ms | 10 Mbps | 160 Mbps |
| 200 ms | 2.5 Mbps | 40 Mbps |
Silly window syndrome
A receiver whose application reads one byte at a time advertises a one byte window, so the sender sends 1 byte in a 40 byte header. The link fills with overhead.
Both ends avoid it. The receiver does not advertise a tiny window, waiting until a worthwhile amount is free. The sender uses Nagle's algorithm, which holds small writes until the outstanding data is acknowledged or a full segment accumulates.
Nagle's algorithm is also worth disabling deliberately for interactive traffic, with TCP_NODELAY. A game or a terminal wants the keystroke sent now, not batched, and this is one of the few options worth setting by hand.