The ONL NPR Tutorial

NPR Tutorial >> Filters, Queues and Bandwidth

TOC

Using Iperf With TCP

Using iperf to generate TCP traffic is not much different than than using it to generate UDP traffic except that the receiver's maximum window size can have a significant impact in the throughput. A TCP receiver notifies the sender how much space is left in its receiver buffer as part of its flow control mechanism. A TCP sender will stop sending new packets if the number of unacknowledged packets will fill up the receiver window. The default server (receiver) window size is 64 KB or about 42 fullsize TCP segments. If we insert a 100 msec delay along a TCP flow's path (discussed later), a 84 KB receiver buffer will limit the throughput to about 5 Mbps since the sender can send out at most 84 KB in one RTT which is 100 msec.

The receiver buffer should be sized to be twice the bandwidth-delay product (BDP) so that the sender can continue to send packets while retransmissions are inflight. Note that the BDP for a 300 Mbps bottleneck and a 100 msec delay is 3.75 MB, well above the 84 KB of default buffering at the receiver. But since all ONL hosts allow receiver windows to be as large as 20 MB, you can ensure that there is enough receiver buffering by using the -w flag. But keep in mind that if you enter "-w 4M", Linux will actually allocate twice the amount you request. So, "-w 4M" should sufficient for a BDP of 3.75 MB. For example, the 3trcvrs-2npr script below starts TCP iperf servers on three NPR 2 hosts and requests receiver buffers of 4 MB (and will be allocated 8 MB of buffering):

1    #!/bin/sh
2    source /users/onl/.topology	# define env vars $n1p2, ...
3    ssh $n2p1 /usr/local/bin/iperf -s -w 4M &
4    ssh $n2p2 /usr/local/bin/iperf -s -w 4M &
5    ssh $n2p3 /usr/local/bin/iperf -s -w 4M &

This script is similar to the 2urcvrs-2npr script we saw earlier (see Generating Traffic With Iperf). Recall that the -s runs iperf in server mode, and the ampersand runs ssh as a background process so that the next command can be started immediately. The only differences are that there is no -u (UDP) flag; it uses a larger receiver buffer flag (-w 4M); and it starts three servers instead of two.

Below is the sender-side script 3tsndrs-2npr which starts three TCP iperf processes staggered by five seconds each by remotely running iperf by logging into hosts $n1p1, $n1p2 and $n1p3:

1    #!/bin/sh
2    source /users/onl/.topology	# define env vars $n2p2, ...
3    ssh $n1p1 /usr/local/bin/iperf -c n2p1 -t 20 &
4    sleep 5
5    ssh $n1p2 /usr/local/bin/iperf -c n2p2 -t 20 &
6    sleep 5
7    ssh $n1p3 /usr/local/bin/iperf -c n2p3 -t 20 &

This script is similar to the 2usndrs-2npr script we saw earlier. Again, the only differences are:

• There is no -u (UDP) flag;
• There is no -b (bandwidth) flag; and
• It starts three clients instead of two.

Recap

• TCP Receiver Window:
  The receiver's (iperf server) socket buffer holds unacknowledged packets. A TCP receiver advertises the amount of buffer space available to the sender. The sender stops sending new packets when the amount of unacknowledged packets can fill the buffer. In Linux, the -w iperf argument sets the size of the receiver buffer to twice the requested amount.

NPR Tutorial >> Filters, Queues and Bandwidth	TOC