IPERF and TCP window size
IPERF and TCP window size
The TCP window size can affect the results of an IPERF test and mislead a user on a network’s performance capabilities.
Iperf appears to use different TCP window sizes depending on the version and OS of the build. The actual implementation of the TCP window for a given OS is beyond the scope of this article, however, it is possible to give Iperf hints about what window size to use/request. I say ‘use/request’ because it it is not clear to me how one verifies the tcp window size actually in use.
In the following example, ‘blue’ has a 5ms round trip time to iperf.wiscnet.net.
[m7h@blue ~]$ iperf -c iperf.wiscnet.net -i 1 -w 8KB ------------------------------------------------------------ Client connecting to iperf.wiscnet.net, TCP port 5001 TCP window size: 8.0 KByte ------------------------------------------------------------ [ 3] local 216.56.13.114 port 37348 connected with 205.213.110.244 port 5001 [ 3] 0.0- 1.0 sec 2.37 MBytes 19.9 Mbits/sec [ 3] 1.0- 2.0 sec 2.30 MBytes 19.3 Mbits/sec [ 3] 2.0- 3.0 sec 2.27 MBytes 19.1 Mbits/sec [ 3] 3.0- 4.0 sec 2.24 MBytes 18.8 Mbits/sec [ 3] 4.0- 5.0 sec 2.31 MBytes 19.4 Mbits/sec [ 3] 5.0- 6.0 sec 2.31 MBytes 19.4 Mbits/sec [ 3] 6.0- 7.0 sec 2.29 MBytes 19.2 Mbits/sec [ 3] 7.0- 8.0 sec 2.26 MBytes 18.9 Mbits/sec [ 3] 8.0- 9.0 sec 2.20 MBytes 18.5 Mbits/sec [ 3] 9.0-10.0 sec 2.23 MBytes 18.7 Mbits/sec [ 3] 0.0-10.0 sec 22.8 MBytes 19.1 Mbits/sec
Here is the same test with a 64KB window
[m7h@blue ~]$ iperf -c iperf.wiscnet.net -i 1 -w 64KB ------------------------------------------------------------ Client connecting to iperf.wiscnet.net, TCP port 5001 TCP window size: 128 KByte (WARNING: requested 64.0 KByte) ------------------------------------------------------------ [ 3] local 216.56.13.114 port 37353 connected with 205.213.110.244 port 5001 [ 3] 0.0- 1.0 sec 17.8 MBytes 149 Mbits/sec [ 3] 1.0- 2.0 sec 17.8 MBytes 149 Mbits/sec [ 3] 2.0- 3.0 sec 18.1 MBytes 152 Mbits/sec [ 3] 3.0- 4.0 sec 18.3 MBytes 153 Mbits/sec [ 3] 4.0- 5.0 sec 18.5 MBytes 155 Mbits/sec [ 3] 5.0- 6.0 sec 18.5 MBytes 155 Mbits/sec [ 3] 6.0- 7.0 sec 18.3 MBytes 154 Mbits/sec [ 3] 7.0- 8.0 sec 18.6 MBytes 156 Mbits/sec [ 3] 8.0- 9.0 sec 18.2 MBytes 153 Mbits/sec [ 3] 9.0-10.0 sec 18.3 MBytes 154 Mbits/sec [ 3] 0.0-10.0 sec 182 MBytes 153 Mbits/sec
8kb * 8 = 64kb. 20 Mbps * 8 = 160 Mbps. This is not a coincidence.
A single iperf flow with a 8KB buffer size is not representative of a production enviornment, which would likely have many active hosts in a single LAN.
If you are trying to optimize TCP throughput for a single flow, increasing packet payload size and TCP windows are your best bets. If you can decrease the round trip time, that is also valuable.
Other considerations:
On 4/18/2013 a commenter chimed in with the following:
> I have done testing with iperf and although the behavior changes with
> the -w flag, i.e. I saw an improvement in bandwidth utilization, packet
> captures show that the hosts are using TCP windowing well outside of the
> requested size. I suspect the -w flag may be controlling an internal
> buffer on the application, but I saw no changes to the TCP windowing.
> Testing was conducted on Windows 7 professional and Red Hat EL Server 5.9.
On 2/10/2015 a commenter chimed in on the following:
> If you modify the window size with the -w switch, you MUST modify the window
> size on both the client and server sides. Interestingly, you do see a significant
> boost in throughput by modifying it on the client side only. This must relate to an
> increased buffer as suggested.
> I have confirmed via Wireshark that the window size remains at the 64KB default
> on Server 2012 if you change the window size to any other value on the client
> side without changing it on the server side. I have then confirmed that changing
> the value on the server side simultaneously results in an increased window size
> as displayed in Wireshark. Between two Hyper-V virtual machines on the same
> server with 10Gb virtual ethernet connections, a window size of 1MB is very
> unstable, rising significantly above and below the specified window size. Perhaps
> an indication of reaching the maximum throughput of the VM’s.
Iperf – кроссплатформенная консольная утилита с открытым исходным кодом, предназначенная для тестирования пропускной способности сети между двумя узлами. Утилита iperf позволяет генерировать нагрузочный TCP и UDP трафик между хостами. С помощью iperf вы можете быстро измерить максимальную пропускную способность сети между сервером и клиентом, провести нагрузочное тестирование канала связи, маршрутизатора, сетевого шлюза (файервола), вашей Ethernet или Wi-Fi сети.
В этой статье мы покажем, как установить и использовать утилиту iPerf для проверки скорости сети в Windows, Linux и VMware ESXi (есть версии iperf для Android, MacOS, RouterOS от MikroTik и других платформ).
Содержание:
- Установка и использование iPerf в Windows
- Установка iPerf в Linux
- Запуск iPerf в VMware ESXi
- Проверка пропускной способности сети с помощью iPerf
Утилита iPerf является кроссплатформенной и не требует установки, достаточно скопировать и запустить ее на двух устройствах, пропускную способность сети между которыми нужно оценить. iPerf работает в режиме клиент-сервер. На первом компьютере утилита iPerf запускается в режиме сервера (ожидает трафик от клиента). На втором компьютере iPerf запускается в режиме клиента, начинает генерировать TCP/UDP трафик и выполнять измерение максимальной скорости передачи данных. В большинстве случаев сейчас используется версия iPerf3 (поддерживает высокоскоростное UDP тестирование, по умолчанию используется порт 5201 TCP/UDP).
Установка и использование iPerf в Windows
Вы можете скачать iperf 3.1 для Windows по ссылке https://iperf.fr/iperf-download.php или версию iperf2. Достаточно скачать архив iperf и распаковать в локальный каталог на диске. Установка утилиты не требуется. В архиве всего два файла: cygwin1.dll и iperf3.exe.
Вы можете скачать архив iPerf и распаковать его на диск с помощью команд PowerShell:
$iPerfZip = "https://iperf.fr/download/windows/iperf-3.1.3-win64.zip"
$TargetFolder = Join-Path $env:TEMP "iperf.zip"$iPerfPath = Join-Path $env:TEMP "iperf"
if (!(Test-Path $iPerfPath))
{ Invoke-WebRequest -Uri $iPerfZip -OutFile $TargetFolder
Expand-Archive -Path $TargetFolder -DestinationPath $iPerfPath
}
Iperf это консольная утилита и для ее запуска нужно использовать командную строку. Откройте командную строку (cmd.exe) и перейдите в каталог с утилитой. Например:
cd c:\tools\iperf
Если вы запустить программу iperf3.exe без параметров, она выведет список доступных опций.
Утилита iPerf может работать в режиме сервера (параметр -s) или клиента (-c). Если вы запускаете iPerf сервер на Windows, нужно открыть входящие порт 5201 для протоколов TCP и UDP. Можно открыть порты через графический интерфейс Windows Defender Firewall или с помощью команд PowerShell. Создайте и включите правила файервола так:
New-NetFirewallRule -DisplayName 'iPerf-Server-Inbound-TCP' -Direction Inbound -Protocol TCP -LocalPort 5201 -Action Allow | Enable-NetFirewallRule
New-NetFirewallRule -DisplayName 'iPerf-Server-Inbound-UDP' -Direction Inbound -Protocol UDP -LocalPort 5201 -Action Allow | Enable-NetFirewallRule
Для Windows есть несколько реализаций графического интерфейса. Например, Iperf3-Cygwin-GUI и jperf.
Утилита jperf написана на Java (для работы на компьютере должна быть установлена Java-машина). Помимо графических рюшечек к CLI интерфейсу, Jperf умеет в реальном времени строить графики пропускной способности канала связи.
Для использования достаточно указать адрес сервера iPerf и запустить проверку.
В галерее скриптов PowerShell есть отдельный модуль iPerfAutomate, который можно использовать для получения данных измерения производительности сети из скриптов PowerShell. Вы можете установить модуль так:
Install-Module -Name iPerfAutomate
Установка iPerf в Linux
В дистрибутивах CentOS/RHEL/Fedora 8 пакет iperf3 включен в состав базового репозитория AppStream (в CentOS 7 iperf3 есть в EPEL). Вы можете установить его стандартной командой yum/dnf:
# dnf install iperf3
В дистрибутивах Debian/Ubuntu вы можете установить утилиту iperf3 командой:
$ sudo apt install iperf3
Если данный Linux сервер планируется использовать в качестве сервере iperf3, нужно открыть порт 5201 в firewalld (или iptables):
# firewall-cmd --permanent --add-port=5201/tcp
# firewall-cmd --permanent --add-port=5201/udp
# firewall-cmd --reload
Запуск iPerf в VMware ESXi
VMware удалила утилиту iPerf в ESXi 6.7, но вернула его в 6.7U1. Проверьте, что на вашем хосте ESXi установлен iPerf. Подключитесь к консоли ESXi по SSH, перейдите в каталог
/usr/lib/vmware/vsan/bin
и проверьте, есть ли в нем файл iperf или iperf3.
Если iPerf не установлен, вы можете вручную скачать offline bundle с iperf здесь (http://vibsdepot.v-front.de/depot/bundles/iperf-2.0.5-1-offline_bundle.zip), скопируйте его на ESXi хост и установить командой:
# esxcli software vib install -d /iperf-2.0.5-1-offline_bundle.zip –no-sig-check
Утилита будет установлена в каталог /opt/iperf/bin и для ее запуска нужно указывать команду:
# /opt/iperf/bin/iperf3
Если вы запускаете iPerf сервер на хосте ESXi, нужно открыть порты 5201. По умолчанию эти порты запрещены в файерволе ESXi. Чтобы открыть их, можно временно отключить файервол:
# esxcli network firewall get# esxcli network firewall set --enabled=false
# esxcli network firewall get
Не забудьте включить файервол ESXi после окончания проверки пропускной способности сети:
# esxcli network firewall set --enabled true
При запуске iPerf в ESXi его нужно привязывать к интерфейсу vmkernel:
# esxcli network ip interface ipv4 get
Проверку доступности между хостами ESXi можно выполнить с помощью:
# vmkping IP address
Запускайте iPerf на полученном IP адресе:
# iperf -s -B 192.168.31.50
Проверка пропускной способности сети с помощью iPerf
Рассмотрим теперь несколько примеров использования iperf для тестирования пропускной способности сети. В этом примере мы будем использовать сервер с CentOS в качестве сервера iperf. Запустим утилиту iperf в серверном режиме:
# iperf3 –s
Сервер iperf запущен, он ожидает соединения на порту TCP/5201.
Server listening on 5201
Важно. Аргументы утилиты iperf регистрозависимы!
Можно запустить iperf сервер с большим размером TCP окна и на другом порту:
# iperf3 -s -w 32768 –p 5203
-w 32768 – зададим размер окна TCP в 32 KB (по умолчанию около 8 Кб)
–p 5203 – порт, на котором ожидает подключения iperf (напоминаю, что iperf2 по умолчанию слушает на порту 5001).
Можно запустить сервер iPerf в режиме службы Windows с помощью ключа -D.
В качестве клиента iperf я использую компьютер с Windows 10. Запустите командную строку и перейдите в каталог с исполняемым файлом iperf:
cd c:\tools\iperf
Чтобы запустить проверку сети со стороны клиента, укажите адрес (имя) хоста, где запущен сервер iperf:
iperf3.exe -c 192.168.1.202
Клиент начнет генерировать сетевой трафик и выполнит тестирование канала в течении 10 секунд и выведет примерно такую таблицу:
- Interval – промежуток тестирования (в сек.);
- Transfer – размер переданных данных за это время;
- Bandwidth – средняя скорость передачи данных.
Если вы запустили сервер iperf с увеличенным размером TCP окна, вы можете использовать следующую команду для получения максимальной нагрузки на сеть:
iperf3.exe -c 192.168.1.202 -P 8 -t 30 -w 32768 -i 5 -f g
- -c 192.168.1.202 – IP адрес сервера iperf;
- -w 32768 — увеличиваем размер TCP окна;
- -t 30 – время в секундах, в течении которого выполняется тестирование (по умолчанию 10 секунд);
- -P 8 — число параллельных потоков (подключений), используется для получения максимальной нагрузки на канал;
- -i 5 – выводить статистику на экран каждые 5 секунд. Параметр удобно использовать при продолжительных тестах (несколько минут, часов);
- -f m — выводить результаты в Мбит/с. Здесь можно использовать атрибуты kmgKMG (килобиты, мегабиты, мегабайты и т.д.).
В нашем примере тестирование длилось 30 секунд. В итоговом отчете нас интересует значения столбца Bandwidth в последней строке [SUM]. Здесь указаны средняя скорость отправки (sender) и получения данных по сети (receiver).
В нашем случае средняя пропускная способность сети между двумя узлами – 79,7 Мбит/с. Было передано 285 Мб данных (столбец Transfer).
Можно запустить iperf в обратном режиме (сервер отправляет данные, а клиент принимает), для этого на клиенте указывается опция –R.
По-умолчанию утилита генерирует TCP трафик, если вам нужно проверить скорость сети для UDP пакетов, необходимо использовать ключ –u (сервер при этом запускается командой:
iperf3 -s –u
).
Если вам нужно проверить ваш интернет-канал (предоставляемый провайдером), можно воспользоваться одним из публичных iperf серверов (список доступен здесь):
iperf3 -c iperf.it-north.net
Обратите внимание, что iperf3 не поддерживает несколько одновременных тестов. Если сервер iperf сейчас выполняет тестирование с одним клиентов, то при попытке подключиться к нему с другого вы получите ошибку: iperf3: error — the server is busy running a test. try again later.
Если нужно оценить пропускную способность сети в обоих направлениях (в дуплексном режиме), дополнительно на клиенте нужно указать опцию –d:
iperf3.exe -c IP -P 8 -t 30 -w 32768 -d
Во время выполнения теста сете с помощью iperf вы можете следить за нагрузкой на сетевой интерфейс компьютера через Task Manager.
Важно отметить, что при тестировании Iperf используем всю доступную пропускную способность канала связи между клиентом и сервером, что может негативно повлиять на продуктивные приложения и пользователей.
Полный список опций утилиты iperf можно получить так:
iperf3 –help
Iperf – простая и удобная сетевая утилита, которая поможет вам измерить производительность сетевого подключения и максимальную скорость передачи данных между двумя устройствами.
Measuring network performance has always been a difficult and unclear task, mainly because most engineers and administrators are unsure which approach is best suited for their LAN or WAN network.
A common (and very simple) method of testing network performance is by initiating a simple file transfer from one end (usually workstation) to another (usually server), however, this method is frequently debated amongst engineers and there is good reason for that: When performing file transfers, we are not only measuring the transfer speed but also hard disk delays on both ends of the stream. It is very likely that the destination target is capable of accepting greater transmission rates than the source is able to send, or the other way around. These bottlenecks, caused by hard disk drives, operating system queuing mechanism or other hardware components, introduce unwanted delays, ultimately providing incorrect results.
The best way to measure the maximum throughput and other aspects of a network is to minimise the delay introduced by the machines participating in the test. High/Mid-end machines (servers, workstations or laptops) can be used to perform these tests, as long as they are not dealing with other tasks during the test operations.
While large companies have the financial resources to overcome all the above and purchase expensive equipment dedicated to testing network environments, the rest of us can rely on other methods and tools, most of which are freely available from the open source community.
Related articles:
- Complete Guide to Netflow: How Netflow & its Components Work. Netflow Monitoring Tools
- Netflow: Monitor Bandwidth & Network Utilization. Detect LAN, WAN, Wi-Fi Bottlenecks, Unusual Traffic Patterns, Problems and more
- Netflow vs SNMP. Two Different Approaches to Network Monitoring
Introducing Iperf
Iperf is a simple and very powerful network tool that was developed for measuring TCP and UDP bandwidth performance. By tuning various parameters and characteristics of the TCP/UDP protocol, the engineer is able to perform a number of tests that will provide an insight into the network’s bandwidth availability, delay, jitter and data loss.
Main features of Iperf include:
- TCP and UDP Bandwidth Measurement
- Reporting of Maximum Segment Size / Maximum Transmission Unit
- Support for TCP Window size
- Multi-threaded for multiple simultaneous connections
- Creation of specific UDP bandwidth streams
- Measurement of packet loss
- Measurement of delay jitter
- Ability to run as a service or daemon
- Option to set and interval to automate performance tests
- Save results and errors to a file (useful for reviewing results later)
- Runs under Windows, Linux OSX or Solaris
Unlike other fancy tools, Iperf is a command line program that accepts a number of different options, making it very easy and flexible to use. Users who prefer GUI based tools can download Kperf or Jperf, which are enhancement projects aimed to provide a friendly GUI interface for Iperf.
Another great thing about Iperf is that both ends do not require to be on the same type of operating system. This means that one end can be running on a Windows PC/Server while the other end is a Linux based system.
Currently supported operating systems are as follows:
- Windows 2000, XP, 2003, Vista, 7, 8 & Windows 2008
- Linux 32bit (i386)
- Linux 64bit (AMD64)
- MacOS X (Intel & PowerPC)
- Oracle Solaris (8, 9 and 10)
Downloading Iperf/Jperf for Windows & Linux — Compiling & Installing on Linux
Iperf is available as a free download from our Administrator Utilities download section. The downloadable zip file contains the Windows and Linux version of Iperf, along with the Java-based graphical interfaces (Jperf). Full installation instructions are available within the .zip file.
The Linux version is easily installed using the procedure outlined below. First step is to untar and unzip the file containing the Iperf application:
[root@Nightsky ~]# tar -zxvf iperf-2.0.5.tar.gz
Next, enter the Iperf directory, configure, compile and install the application:
[root@Nightsky ~]# cd iperf-2.0.5
[root@Nightsky iperf-2.0.5]# ./configure
[root@Nightsky iperf-2.0.5]# make
<output omitted>
[root@Nightsky iperf-2.0.5]# make install
<output omitted>
Finally, clean the directory containing our compiled leftover files:
[root@Nightsky iperf-2.0.5]# make clean
Iperf can be conveniently found in the /usr/local/bin/ directory on the Linux server or workstation.
Below is a screenshot from the Windows GUI — Jperf application. Its friendly interface makes it easy to select bandwidth speed, protocol specific parameters, and much more, with just a few clicks. At the top of the GUI, Jperf will also display the CLI command used for the options selected — a neat feature:
Ideas On Unleashing Iperf – Detailed Examples On How To Use Iperf
Having a great tool like Iperf to measure network performance, packet loss, jitter and other characteristics of a network, opens a number of brilliant possibilities that can help an engineer not only identify possible pitfalls in their network (LAN or WAN), but also test different vendor equipment and technologies to discover real performance differences between them.
Here are a few ideas the Firewall.cx team came up with during our brainstorming session on Iperf:
- Measuring the network (LAN) backbone throughput.
- Measuring Jitter and packet loss across links. The jitter value is particularly important on network links supporting voice over IP (VoIP) because a high jitter can break a VoIP call.
- Test WAN link speeds and CIR – Is the Telco provider delivering the speeds we are paying for?
- Test router or firewall VPN throughput between links. By tuning IPSec encryption algorithms we can increase our throughput significantly.
- Test Access Point performance between clients. Wireless clients connect at 150Mbps or 300Mbps to an access point, but what are the maximum speeds that can be achieved between them?
- Test Client – Server bottlenecks. If there’s a server performance issue and we are not quite sure if its network related, Iperf can help shed light on the source of the problem, leaving out of the equation possible bottlenecks such as hard disk drives.
- Creating parallel data transfer streams to increase load on the network to test router or switch utilisation. By running Iperf on multiple workstations with multiple threads, we can create a significant amount of load on our network and perform various stress-tests.
At first sight, it is evident that Iperf is a tool that can be used to test any part of your network, whether it be Copper (UTP) links, fiber optic links, Wi-Fi, leased lines, VoIP infrastructure and much more.
Because every network has different needs and problems we thought it would be better to take a different approach to Iperf and, instead of presenting test results of our setups (LAB Environment), show how it can be used to test and diagnose different problems engineers are forced to deal with.
By having a firm understanding how to use the options supported by Iperf, engineers can tweak the commands to help them identify their own network problems and test their network performance.
For this reason, we have split this Iperf presentation by covering its various parameters. Note the parameters are case sensitive:
- Default Iperf Settings for Server and Client
- Communications Ports (-p), Interval (-i) and timing (-t)
- Data format report (Kbps, Mbps, Kbytes, Mbytes) (-f)
- Buffer lengths to read or write (-l)
- UDP Protocol Tests (-u) & UDP bandwidth settings (-b)
- Multiple parallel threads (-P)
- Bi-directional bandwidth measurement (-r)
- Simultaneous bi-directional bandwidth measurement (-d)
- TCP Window size (-w)
- TCP Maximum Segment Size (MSS) (-M)
- Iperf Help (-h)
Default Iperf Settings for Server and Client
Server Side
By default, Iperf server listens on TCP port 5001 with a TCP window size of 85Kbytes. When running Iperf in server mode under Windows, the TCP window size is set to 64Kbytes. The Iperf server is run using the following command:
[root@Nightsky bin]# iperf -s
————————————————————
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
————————————————————
Client Side
The Iperf client connects to the Iperf server at TCP port 5001. When running in client mode we must specify the Iperf server’s IP address. Iperf will run immediately and present its results:
C:\Users\Chris\Desktop\iperf-2.0.5-2-win32> iperf -c 192.168.5.5
————————————————————
Client connecting to 192.168.5.5, TCP port 5001
TCP window size: 64.0 KByte (default)
————————————————————
[ 3] local 192.168.5.237 port 52339 connected with 192.168.5.5 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 105 MBytes 87.6 Mbits/sec
The average bandwidth test was 87.6Mbps
Server Side Results
The server also provides the test results, allowing both ends to verify the results. In some cases there might be a minor difference in the bandwidth because of how it’s calculated from each end:
————————————————————
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
————————————————————
[ 4] local 192.168.5.5 port 5001 connected with 192.168.5.237 port 52339
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 105 MBytes 87.5 Mbits/sec
Communications Ports (-p), Interval (-i) and Timing (-t)
The port under which Iperf runs can be changed using the –p parameter. The same value must be configured on both server and client side. The interval -i is a Server/Client parameter used to set the interval between periodic bandwidth reports, in seconds, and is very useful to see how bandwidth reports change during the testing period.
The timing parameter –t is client specific and specifies the duration of the test in seconds. The default is 10 seconds.
Server Side
[root@Nightsky bin]# iperf -s -p 32000
————————————————————
Server listening on TCP port 32000
TCP window size: 85.3 KByte (default)
————————————————————
Client Side
C:\Users\Chris\Desktop\iperf-2.0.5-2-win32> iperf -c 192.168.5.5 -p 32000 -i 2 -t 5
————————————————————
Client connecting to 192.168.5.5, TCP port 32000
TCP window size: 64.0 KByte (default)
————————————————————
[ 3] local 192.168.5.237 port 52602 connected with 192.168.5.5 port 32000
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 2.0 sec 20.4 MBytes 85.5 Mbits/sec
[ 3] 2.0- 4.0 sec 20.8 MBytes 87.0 Mbits/sec
[ 3] 0.0- 5.0 sec 51.8 MBytes 86.5 Mbits/sec
Server Side Results
————————————————————
Server listening on TCP port 32000
TCP window size: 85.3 KByte (default)
————————————————————
[ 4] local 192.168.5.5 port 32000 connected with 192.168.5.237 port 52678
[ ID] Interval Transfer Bandwidth
[ 4] 0.0- 5.0 sec 51.6 MBytes 86.2 Mbits/sec
Data Format Report (Kbytes & Kbps, Mbytes & Mbps) (-f) – Server/Client Parameter
Iperf can display the bandwidth results in different format, making it easy to read. Bandwidth measurements and data transfers will be displayed in the format selected.
Server side
[root@Nightsky bin]# iperf -s
————————————————————
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
————————————————————
Client Side
Here a test is performed on a 10Mbps link using default parameters. Notice the Transfer and Bandwidth report at the end:
C:\Users\Chris\Desktop\iperf-2.0.5-2-win32> iperf -c 192.168.5.5
————————————————————
Client connecting to 192.168.5.5, TCP port 5001
TCP window size: 64.0 KByte (default)
————————————————————
[ 3] local 192.168.5.237 port 53006 connected with 192.168.5.5 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.2 sec 11.4 MBytes 9.39 Mbits/sec
Same test was executed with the –f k parameter so that Iperf would display the results in Kilobytes and Kbps format:
C:\Users\Chris\Desktop\iperf-2.0.5-2-win32> iperf -c 192.168.5.5 -f k
————————————————————
Client connecting to 192.168.5.5, TCP port 5001
TCP window size: 64.0 KByte (default)
————————————————————
[ 3] local 192.168.5.237 port 53038 connected with 192.168.5.5 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.2 sec 11648 KBytes 9373 Kbits/sec
Buffer Lengths To Read Or Write (-l) – Server/Client Parameter
The buffer lengths are rarely used, however, they are useful when dealing with large capacity links such as local networks (LAN). The –l parameter specifies the length of buffer read/write for each side and is a client/server parameter. Values specified can be in K (Kbytes) or M (Mbytes). It’s best to always ensure both sides have the same buffer value set. The default length of read/write buffer is 8K.
Server Side
[root@Nightsky bin]# iperf -s -l 256K
————————————————————
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
————————————————————
Client Side with default read/write buffer of 8K.
Note that for test, the Server side was not set, making it the default value of 8K.
C:\Users\Chris\Desktop\iperf-2.0.5-2-win32> iperf -c 192.168.5.241
————————————————————
Client connecting to 192.168.5.241, TCP port 5001
TCP window size: 64.0 KByte (default)
————————————————————
[ 3] local 192.168.5.237 port 53331 connected with 192.168.5.241 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 735 MBytes 616 Mbits/sec
Client Side with read/write buffer of 256K.
Note that, for this test, the Server side was set to the same buffer length value of 256K.
C:\Users\Chris\Desktop\iperf-2.0.5-2-win32> iperf -c 192.168.5.241 -l 256K
————————————————————
Client connecting to 192.168.5.241, TCP port 5001
TCP window size: 64.0 KByte (default)
————————————————————
[ 3] local 192.168.5.237 port 53330 connected with 192.168.5.241 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 796 MBytes 667 Mbits/sec
Client Side with read/write buffer of 20MB.
Note that, for this test, the Server side was set to the same buffer length value of 20MB. Notice the dramatic increase of Transfer and Bandwidth with a 20MB read/write buffer:
C:\Users\Chris\Desktop\iperf-2.0.5-2-win32> iperf -c 192.168.5.241 -l 20M
————————————————————
Client connecting to 192.168.5.241, TCP port 5001
TCP window size: 64.0 KByte (default)
————————————————————
[ 3] local 192.168.5.237 port 53860 connected with 192.168.5.241 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.2 sec 980 MBytes 803 Mbits/sec
When running tests with large read/write buffers it is equally interesting to monitor the client’s or server’s CPU, memory and bandwidth usage.
Since the 20MB buffer is swapped to memory during the test there will be a noticeable increase of memory usage. Those curious can also try a much larger buffer such as 100MB to see how the system will respond. At the same time, CPU usage will also increase as it is handing the packets being generated and received. Our Dual-Core CPU handled the test without a problem, however, it doesn’t take much to bring the system to its knees. For this reason it is highly advisable not run other heavy applications during the tests:
On the other hand, monitoring the network utilisation through the Windows Task Manager also helps provide a visual result of the network throughput test:
UDP Protocol Tests (-u) & UDP Bandwidth Settings (-b) – Important For VoIP Networks
The –u parameter is a Server/Client specific parameter.
VoIP networks are great candidates for this type of test and extremely important. UDP tests can provide us with valuable information on jitter and packet loss. Jitter is the latency variation and does not depend on the latency itself. We can have high response times and low jitter values without introducing VoIP communications problems. High jitter can cause serious problems to VoIP calls and even break them.
The UDP test also measures the packet loss of your network. A good quality link must have a packet loss less than 1%.
The –b parameter is client specific and allows us to specify the bandwidth to send in bits/sec. The useful combination of –u and –b allows us to control the rate at which data is sent across the link being tested. The default value is 1Mbps.
Server Side
[root@Nightsky bin]# iperf -s -u
————————————————————
Server listening on UDP port 5001
Receiving 1470 byte datagrams
UDP buffer size: 224 KByte (default)
————————————————————
Client Side
The following command instructs our client to send UDP data at the rate of 10Mbps:
C:\Users\Chris\Desktop\iperf-2.0.5-2-win32> iperf -c 192.168.5.5 -u -b10m
————————————————————
Client connecting to 192.168.5.5, UDP port 5001
Sending 1470 byte datagrams
UDP buffer size: 64.0 KByte (default)
————————————————————
[ 3] local 192.168.5.237 port 64214 connected with 192.168.5.5 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 11.8 MBytes 9.89 Mbits/sec
[ 3] Sent 8418 datagrams
[ 3] Server Report:
[ 3] 0.0-10.0 sec 5.23 MBytes 4.39 Mbits/sec 0.218 ms 4683/ 8417 (56%)
[ 3] 0.0-10.0 sec 1 datagrams received out-of-order
It is important to note that the Iperf client presents its local and remote Iperf server statistics. While the client reports that it was able to send data at the rate of 9.89Mbps, the server reported it was receiving data at the rate of 4.39Mbps, clearly indicating a problem in our link.
Next in the server’s bandwidth report (4.39Mbps) are the jitter and packet loss statistics. The jitter was measured at 0.218msec – an acceptable delay, however, the 56% packet loss is totally unacceptable and explains why the server received slightly less than half (4.39Mbps) of the transmitted rate of 9.89Mbps.
When tests reveal possible network problems it is always best to re-run the test to determine if packet loss is constant or happens at specific times during the total transfer. This information can be revealed by repeating the Iperf command but including the –i 2 parameter, which instructs our client to send UDP data at the rate of 10Mbps and sets interval between periodic bandwidth reports to 2 seconds:
Client Side
C:\Users\Chris\Desktop\iperf-2.0.5-2-win32> iperf -c 192.168.5.5 -u -b10m -i 2
————————————————————
Client connecting to 192.168.5.5, UDP port 5001
Sending 1470 byte datagrams
UDP buffer size: 64.0 KByte (default)
————————————————————
[ 3] local 192.168.5.237 port 64609 connected with 192.168.5.5 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 2.0 sec 2.32 MBytes 9.74 Mbits/sec
[ 3] 2.0- 4.0 sec 2.40 MBytes 10.1 Mbits/sec
[ 3] 4.0- 6.0 sec 2.34 MBytes 9.80 Mbits/sec
[ 3] 6.0- 8.0 sec 2.07 MBytes 8.68 Mbits/sec
[ 3] 8.0-10.0 sec 2.06 MBytes 8.64 Mbits/sec
[ 3] 0.0-10.3 sec 11.2 MBytes 9.10 Mbits/sec
[ 3] Sent 7983 datagrams
[ 3] Server Report:
[ 3] 0.0-50.4 sec 4.76 MBytes 793 Kbits/sec 0.270 ms 4584/ 7982 (57%)
[ 3] 0.0-50.4 sec 1 datagrams received out-of-order
The results with 2 second interval reporting show that there was a significant drop in transmission speed a bit later than half way through the test, between 6 and 10 seconds. If this was a leased line or Frame Relay link, it would most likely indicate that we are hitting our CIR (Committed Information Rate) and the service provider is slowing down our transmission rates.
Of course, further testing is needed, but any engineer can appreciate the valuable information provided with this simple test.
Multiple Parallel Threads (-P) — Client Specific Parameter
The multiple parallel thread parameter –P is client specific and allows the client side to run multiple threads at the same time. Obviously, using this parameter would divide the bandwidth to the amount of threads running and it’s considered a valuable parameter when testing QoS functionality. We combined it with the –l 4M parameter to increase the read/write buffer to 4MB, increasing the performance on both ends.
Server Side
[root@Nightsky bin]# iperf -s -l 4M
————————————————————
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
————————————————————
Client Side
C:\Users\Chris\Desktop\iperf-2.0.5-2-win32> iperf -c 192.168.5.5 -l 4M -P 3
————————————————————
Client connecting to 192.168.5.5, TCP port 5001
TCP window size: 64.0 KByte (default)
————————————————————
[ 5] local 192.168.5.237 port 54222 connected with 192.168.5.5 port 5001
[ 3] local 192.168.5.237 port 54220 connected with 192.168.5.5 port 5001
[ 4] local 192.168.5.237 port 54221 connected with 192.168.5.5 port 5001
[ ID] Interval Transfer Bandwidth
[ 5] 0.0-11.5 sec 44.0 MBytes 32.1 Mbits/sec
[ 4] 0.0-11.7 sec 44.0 MBytes 31.5 Mbits/sec
[ 3] 0.0-11.8 sec 44.0 MBytes 31.4 Mbits/sec
[SUM] 0.0-11.8 sec 132 MBytes 94.1 Mbits/sec
Individual Bi-directional Bandwidth Measurement (-r) — Client Specific Parameter
The bi-directional parameter –r forces an individual bi-directional test, forcing the client to become the server after its initial test is complete. This option is considered very useful when it is necessary to test the performance in both directions and saves us manually switching the roles between the client and server.
Server Side
[root@Nightsky bin]# iperf -s
————————————————————
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
————————————————————
Client Side
C:\Users\Chris\Desktop\iperf-2.0.5-2-win32> iperf -c 192.168.5.5 -r
————————————————————
Server listening on TCP port 5001
TCP window size: 64.0 KByte (default)
————————————————————
————————————————————
Client connecting to 192.168.5.5, TCP port 5001
TCP window size: 64.0 KByte (default)
————————————————————
[ 4] local 192.168.5.237 port 54538 connected with 192.168.5.5 port 5001
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 103 MBytes 86.3 Mbits/sec
[ 4] local 192.168.5.237 port 5001 connected with 192.168.5.5 port 39426
[ 4] 0.0-10.0 sec 110 MBytes 92.5 Mbits/sec
Notice the two connections created, one for each direction. A similar report is generated on the server’s side.
Simultaneous Bi-directional Bandwidth Measurement (-d) – Client Specific
The simultaneous bi-directional bandwidth measurement parameter –d is client specific and forces a simultaneous two way data transfer test. Think about is as a full-duplex test between the server and client. This test is great for leased line WAN links which offer synchronous download/upload speeds.
We tested it between our Linux server and Windows 7 client using the –l 5M parameter, to increase the send/receive buffer and test out speeds through a 100Mbit link.
Server Side
[root@Nightsky bin]# iperf -s -l 5M
————————————————————
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
————————————————————
Client Side
C:\Users\Chris\Desktop\iperf-2.0.5-2-win32> iperf -c 192.168.5.5 -d -l 5M
————————————————————
Server listening on TCP port 5001
TCP window size: 64.0 KByte (default)
————————————————————
————————————————————
Client connecting to 192.168.5.5, TCP port 5001
TCP window size: 64.0 KByte (default)
————————————————————
[ 4] local 192.168.5.237 port 52671 connected with 192.168.5.5 port 5001
[ 5] local 192.168.5.237 port 5001 connected with 192.168.5.5 port 39430
[ ID] Interval Transfer Bandwidth
[ 5] 0.0-10.3 sec 90.0 MBytes 73.2 Mbits/sec
[ 4] 0.0-10.7 sec 115 MBytes 90.0 Mbits/sec
We can see the two sessions [4 & 5] created between our two endpoints along with their results – an average of 81,6Mbps ( (73.2+90) / 2), falling slightly short of our expectations of our 100Mbps test link.
TCP Window Size (-w) – Server/Client Parameter
The TCP Window size can be set using the –w parameter. The TCP Window size represents the amount of data that can be sent from the server without the receiver being required to acknowledge it. Typical values are between 2 and 65,535bytes. The default value is 64KB.
Firewall.cx has covered the TCP Window size concept in great depth. Readers can refer to our TCP Windows Size article to understand its importance and how it can help increase throughput on links with increased latency e.g Satellite links.
Server Side
On Linux, when specifying a TCP Window size, the kernel allocated double that requested. Ironically, the Windows operating system allowed a 1MB and even 5MB window size without any problem.
[root@Nightsky bin]# iperf -s -l 5M -w 4000
————————————————————
Server listening on TCP port 5001
TCP window size: 7.81 KByte (WARNING: requested 3.91 KByte)
————————————————————
Client Side
C:\Users\Chris\Desktop\iperf-2.0.5-2-win32> iperf -c 192.168.5.5 -l 5M -w 4000
————————————————————
Client connecting to 192.168.5.5, TCP port 5001
TCP window size: 3.91 KByte
————————————————————
[ 3] local 192.168.5.237 port 54172 connected with 192.168.5.5 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-11.5 sec 55.0 MBytes 40.1 Mbits/sec
Using a 4KB TCP Window size gave us only 40.1Mbps — half of our potential 100Mbps link. When we increased this to 64KB, we managed to squeeze out 93.9Mbps throughput!
TCP Maximum Segment Size (MSS) (-M) — Server/Client Parameter
The Maximum Segment Size (mss) is the largest amount of data, in bytes, that a computer can support in a single unfragmented TCP segment. Readers interested in understanding the importance of mss and how it works can refer to our TCP header analysis article.
If the MSS is set too low or high it can greatly affect network performance, especially over WAN links.
Below are some default values for various networks:
Ethernet – Lan: 1500 Bytes
PPPoE ADSL: 1492 Bytes
Dialup: 576 Bytes
Server Side
[root@Nightsky bin]# iperf -s
————————————————————
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
————————————————————
Client Side
C:\Users\Chris\Desktop\iperf-2.0.5-2-win32> iperf -c 192.168.5.5 -M 1350
WARNING: attempt to set TCP maximum segment size to 1350, but got 1281
————————————————————
Client connecting to 192.168.5.5, TCP port 5001
TCP window size: 64.0 KByte (default)
————————————————————
[ 3] local 192.168.5.237 port 54877 connected with 192.168.5.5 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 105 MBytes 88.2 Mbits/sec
Iperf Help –(h)
While we’ve covered most of the Iperf supported parameters, there are still more readers can discover and work with. Using the iperf –h command will reveal all available options:
C:\Users\Chris\Desktop\iperf-2.0.5-2-win32> iperf -h
Usage: iperf [-s|-c host] [options]
iperf [-h|—help] [-v|—version]
Client/Server:
-f, —format [kmKM] format to report: Kbits, Mbits, KBytes, MBytes
-i, —interval # seconds between periodic bandwidth reports
-l, —len #[KM] length of buffer to read or write (default 8 KB)
-m, —print_mss print TCP maximum segment size (MTU — TCP/IP header)
-o, —output output the report or error message to this specified file
-p, —port # server port to listen on/connect to
-u, —udp use UDP rather than TCP
-w, —window #[KM] TCP window size (socket buffer size)
-B, —bind bind to, an interface or multicast address
-C, —compatibility for use with older versions does not sent extra msgs
-M, —mss # set TCP maximum segment size (MTU — 40 bytes)
-N, —nodelay set TCP no delay, disabling Nagle’s Algorithm
-V, —IPv6Version Set the domain to IPv6
Server specific:
-s, —server run in server mode
-U, —single_udp run in single threaded UDP mode
-D, —daemon run the server as a daemon
Client specific:
-b, —bandwidth #[KM] for UDP, bandwidth to send at in bits/sec
(default 1 Mbit/sec, implies -u)
-c, —client run in client mode, connecting to
-d, —dualtest Do a bidirectional test simultaneously
-n, —num #[KM] number of bytes to transmit (instead of -t)
-r, —tradeoff Do a bidirectional test individually
-t, —time # time in seconds to transmit for (default 10 secs)
-F, —fileinput input the data to be transmitted from a file
-I, —stdin input the data to be transmitted from stdin
-L, —listenport # port to receive bidirectional tests back on
-P, —parallel # number of parallel client threads to run
-T, —ttl # time-to-live, for multicast (default 1)
-Z, —linux-congestion set TCP congestion control algorithm (Linux only)
In this article we showed how IT Administrators, IT Managers and Network Engineers can use IPerf to correctly test their network throughput, network delay, packet loss and link reliability.
Обновлено:
Опубликовано:
Тематические термины: iPerf, Linux, CentOS, Windows
iPerf является кроссплатформенным приложением и может быть установлен на любую популярную операционную систему. В данной инструкции будет рассмотрена установка сервера на Linux CentOS и клиента как на последнем, так и Microsoft Windows.
Установка и запуск
Запуск клиента и проверка сервера
Ключи
Автозапуск
Примеры
Версия для Windows
iPerf и iPerf3
Публичные сервера
Свой публичный сервер
Возможные проблемы
Установка сервера/клиента
По сути, нет отдельного iPerf для сервера или клиента — это один и тот же программный продукт, который может запускаться в режиме сервера или выполнять клиентские команды.
Для начала выполняем установку расширенного репозитория:
yum install epel-release
Устанавливаем iPerf:
yum install iperf3
Открываем порт в брандмауэре:
firewall-cmd —permanent —add-port=5201/tcp
firewall-cmd —permanent —add-port=5201/udp
firewall-cmd —reload
* в данном примере мы открыли порт 5201 (для iPerf по умолчанию) для пакетов TCP и UDP.
Запускаем сервер:
iperf3 -s
На экране появится:
————————————————————
Server listening on 5201
————————————————————
Сервер ждет запросов.
Клиентские запросы и проверка сервера
Напомню, что клиент может быть установлен на любую систему. Также, как в первом случае, выполняем установку клиента на другой компьютер в сети.
После можно сразу выполнить команду:
iperf3 -c 192.168.0.15
* где 192.168.0.15 — iPerf сервер.
На клиенте мы увидим что-то подобное:
[ 4] local 192.168.0.20 port 47068 connected to 192.168.0.15 port 5201
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 104 MBytes 873 Mbits/sec 10 391 KBytes
[ 4] 1.00-2.00 sec 110 MBytes 921 Mbits/sec 4 393 KBytes
[ 4] 2.00-3.00 sec 111 MBytes 928 Mbits/sec 1 478 KBytes
[ 4] 3.00-4.00 sec 104 MBytes 875 Mbits/sec 5 423 KBytes
* где:
- 192.168.0.20 — адрес клиента
- 192.168.0.15 — адрес сервера
- ID — идентификатор запросов, нужен для ориентирования, если к серверу идет несколько обращений.
- Interval — промежуток времени в секундах, на протяжении которого выполнялась передача данных.
- Transfer — сколько было передано данных за интервал времени.
- Bandwidth — средняя скорость передачи данных за интервал времени.
- Retr — количество повторно отправленных TCP-сегментов.
- Cwnd — одновременно переданных данных.
Все ключи запуска iPerf
Общие для сервера и клиента:
| Ключ | Описание |
|---|---|
| -p | Определить порт, на котором будет слушать сервер или отправлять запросы клиент |
| -f | Формат отчетов — kmgKMG (Kbits, Mbits, KBytes, MBytes, …) |
| -i | Задать интервал, в течение которого выполняется одна проверка |
| -F | Указать файл, из которого будут взяты входные данные для запуска |
| -A | Степень нагрузки на процессор |
| -B | Указать, через какой сетевой интерфейс работать |
| -V | Детализированные сообщения в консоли |
| -J | Вывод в формате json |
| —logfile | Весь вывод в отдельный лог-файл |
| -d | Режим отладки (много сообщений) |
| -v | Показать версию программы |
| -h | Вызвать справку по работе с программой |
Для сервера:
| Ключ | Описание |
|---|---|
| -s | Запустить iPerf в режиме сервера |
| -D | Запустить как демона (как службу) |
| -I | Указать pid-файл |
| -1 | Принять запрос от одного клиента и завершить работу |
Для клиента:
| Ключ | Описание |
|---|---|
| -c | Запустить iPerf в режиме клиента |
| -u | Отправлять UDP-пакеты |
| -b | Формат отчетов для bandwidth (средней скорости) |
| -t | Количество секунд, в течение которых будет идти проверка скорости |
| -n | Объем данных для проверки (применяется вместо времени -t) |
| -k | Количество пакетов для проверки (вместо -t или -n) |
| -l | Длина буфера записи/чтения |
| -P | Число параллельных запросов |
| -R | Обратный режим — сервер отправляет, клиент принимает |
| -w | Размер сетевого окна |
| -C | Установить алгоритм управления перегрузкой TCP |
| -M | Задать максимальный размер MTU |
| -4 | Работать только для IPv4 |
| -6 | Работать только для IPv6 |
| -Z | Использовать метод «нулевой копии» для отправки данных |
| -O | Опустить первые n секунд |
| -T | Задать префикс для каждой строки вывода |
Также самый свежий список ключей можно получить командой:
man iperf3
Автозапуск сервера (создание сервиса в systemd)
По умолчанию, программу нужно запускать вручную. Если мы хотим, чтобы сервер запускался автоматически и работал как служба systemd, выполняем инструкцию ниже.
Создаем юнит в systemd со следующим содержимым:
vi /etc/systemd/system/iperfd.service
[Unit]
Description=iPerf Service
After=network.target
[Service]
Type=forking
PIDFile=/run/iperf3.pid
ExecStart=-/bin/iperf3 -s -D -I /run/iperf3.pid
ExecReload=/bin/kill -HUP $MAINPID
Restart=always
[Install]
WantedBy=multi-user.target
* где
- Description — описание юнита;
- After указывает на юнит, после которого может загружаться наш сервис;
- Type — тип службы;
- PIDFile — путь к pid файлу, в котором хранится номер процесса;
- ExecStart — команда, которую нужно выполнить при старте сервиса (в данном примере запускается iPerf в режиме сервера как демон и создает pid-файл);
- ExecReload — команда для перезапуска службы;
- Restart=always — опция, позволяющая автоматически перезапускать сервис, если он перестанет работать;
- опция WantedBy=multi-user.target позволяет установить для автозапуска службу в обычном многопользовательском режиме.
Перезапускаем systemd:
systemctl daemon-reload
Разрешаем созданный сервис:
systemctl enable iperfd
Запускаем его:
systemctl start iperfd
Проверяем:
systemctl status iperfd
Примеры использования iPerf
Рассмотрим некоторые команды запросов к серверу для проверки скорости соединения.
Использование UDP
iperf3 -c 192.168.0.15 -u
* сам сервер не нужно запускать в UDP-режиме, так как он принимает любые запросы.
Альтернативные порты
Для этого необходимо сначала запустить сервер на нужном порту:
iperf3 -s -p 443
* кстати, можно запустить несколько процессов iperf одновременно, которые будут слушать на разных портах.
* стоит не забывать по настройки брандмауэра. В данном примере понадобиться ввести команды firewall-cmd —permanent —add-port=443/tcp и firewall-cmd —reload.
Теперь можно запускать клиента:
iperf3 -c 192.168.0.15 -p 443
Проверка скорости в течение 30 секунд с интервалами по 2 секунды
iperf3 -c 192.168.0.15 -t 30 -i 2
Несколько параллельных запросов
iperf3 -c 192.168.0.15 -P 3
Отправляем на проверку 3 Гб данных
iperf3 -c 192.168.0.15 -n 3G
Клиент/сервер для Windows
Скачиваем iPerf под Windows с официального сайта. Распаковываем архив и запускаем командную строку (cmd). Переходим в распакованную папку (команда cd).
Можем работать с iperf. Команды такие же, как для Linux, например:
iperf3.exe -c 192.168.0.15
GUI
Для работы с iPerf в графическом интерфейсе есть различные утилиты, например Iperf3-Cygwin-GUI. Скачиваем архив, распаковываем его и запускаем iperf3cygwingui.bat.
В открывшемся окне для простой проверки нужно только прописать адрес iPerf-сервера и нажать кнопку Run Iperf3:
Однако, на моей практике, программа не заработала, а после запуска зависала (или не запускался iperf3.exe). Проблему удалось решить с помощью скачанного с официального сайта клиента (файлов iperf3.exe и cygwin1.dll), которыми я заменил одноименные файлы в папке bin.
При желании, программу можно также запустить в режиме сервера:
jPerf
Для работы в графическом интерфейсе также есть популярная программа jperf. Ее можно скачать с сайта sourceforge. Однако, она создана с использованием старой версии iperf, поэтому я не стал уделять ей много внимания. В целом, процесс и внешний вид не сильно отличается от Iperf3-Cygwin-GUI.
Совместимость iPerf и iPerf3
Клиент и сервер совместимы, но нужно знать, что по умолчанию, рабочий порт для iPerf — 5001, для iPerf3 — 5201.
Поэтому либо необходимо запускать сервер:
iperf3 -s -p 5001
* также не забываем настроить брандмауэр.
Либо запускаем клиентские запросы командой:
iperf -c 192.168.0.15 -p 5201
Публичные сервера iPerf
Актуальные публичные сервера iPerf можно найти на официальном сайте программы. Внимательно смотрите на описания серверов и рабочие порты.
Пример проверки с использованием сервера во Франции:
iperf3 -c bouygues.testdebit.info -p 5200
или в Индонезии:
iperf3 -c iperf.biznetnetworks.com
Свой публичный сервер
Сервер iPerf может обрабатывать только одно подключение одновременно на порту. Поэтому, в условиях публичного использования необходим его запуск на множестве портов одновременно.
Мы настроим сервер при помощи systemd. Ранее мы уже создавали один юнит для запуска iperf. Повторяем процедуру с небольшими изменениями.
Создаем юнит со следующим содержимым:
vi /etc/systemd/system/iperfd5205.service
[Unit]
Description=iPerfService on port %i
After=network.target
[Service]
Type=forking
PIDFile=/var/run/iperf3.5205.pid
ExecStart=-/bin/iperf3 -s -p 5205 -D -I /var/run/iperf3.5205.pid
ExecReload=/bin/kill -HUP $MAINPID
Restart=always
RuntimeMaxSec=3600
[Install]
WantedBy=multi-user.target
* если сравнить с ранее созданным юнитом, мы добавили запуск сервера на порту 5205.
Перезапускаем systemd:
systemctl daemon-reload
Разрешаем юнит и запускаем сервис:
systemctl enable iperfd5205
systemctl start iperfd5205
Повторяем шаги для других портов — создаем для каждого отдельный юнит в systemd. Для публичного сервера лучше создать побольше.
Возможные ошибки
the server is busy running a test. try again later
В данный момент сервер обрабатывает другой запрос или он завис. Если мы являемся администратором сервера, перезагружаем его, в противном случае, ждем.
iPerf — The ultimate speed test tool for TCP, UDP and SCTPTest the limits of your network + Internet neutrality test
Table of contents :
- Change between iPerf 2.0, iPerf 3.0 and iPerf 3.1
- iPerf 3 user documentation
- Change between iPerf 2.0.6, iPerf 2.0.7 and iPerf 2.0.8
- iPerf 2 user documentation
Change between iPerf 2.0, iPerf 3.0 and iPerf 3.1
- iPerf2 features currently supported by iPerf3 :
- TCP and UDP tests
- Set port (-p)
- Setting TCP options: No delay, MSS, etc.
- Setting UDP bandwidth (-b)
- Setting socket buffer size (-w)
- Reporting intervals (-i)
- Setting the iPerf buffer (-l)
- Bind to specific interfaces (-B)
- IPv6 tests (-6)
- Number of bytes to transmit (-n)
- Length of test (-t)
- Parallel streams (-P)
- Setting DSCP/TOS bit vectors (-S)
- Change number output format (-f)
- New Features in iPerf 3.0 :
- Dynamic server (client/server parameter exchange) – Most server options from iPerf2 can now be dynamically set by the client
- Client/server results exchange
- A iPerf3 server accepts a single client simultaneously (multiple clients simultaneously for iPerf2)
- iPerf API (libiperf) – Provides an easy way to use, customize and extend iPerf functionality
- -R, Reverse test mode – Server sends, client receives
- -O, —omit N : omit the first n seconds (to ignore TCP slowstart)
- -b, —bandwidth n[KM] for TCP (only UDP for IPERF 2): Set target bandwidth to n bits/sec (default 1 Mbit/sec for UDP, unlimited for TCP).
- -V, —verbose : more detailed output than before
- -J, —json : output in JSON format
- -Z, —zerocopy : use a ‘zero copy’ sendfile() method of sending data. This uses much less CPU.
- -T, —title str : prefix every output line with this string
- -F, —file name : xmit/recv the specified file
- -A, —affinity n/n,m : set CPU affinity (cores are numbered from 0 — Linux and FreeBSD only)
- -k, —blockcount #[KMG] : number of blocks (packets) to transmit (instead of -t or -n)
- -4, —version4 : only use IPv4
- -6, —version6 : only use IPv6
- -L, —flowlabel : set IPv6 flow label (Linux only)
- -C, —linux-congestion : set congestion control algorithm (Linux and FreeBSD only) (-Z in iPerf2)
- -d, —debug : emit debugging output. Primarily (perhaps exclusively) of use to developers.
- -s, —server : iPerf2 can handle multiple client requests. iPerf3 will only allow one iperf connection at a time.
- New Features in iPerf 3.1 :
- -I, —pidfile file write a file with the process ID, most useful when running as a daemon.
- —cport : Specify the client-side port.
- —sctp use SCTP rather than TCP (Linux, FreeBSD and Solaris).
- —udp-counters-64bit : Support very long-running UDP tests, which could cause a counter to overflow
- —logfile file : send output to a log file.
- iPerf2 Features Not Supported by iPerf3 :
- Bidirectional testing (-d / -r)
- Data transmitted from stdin (-I)
- TTL : time-to-live, for multicast (-T)
- Exclude C(connection) D(data) M(multicast) S(settings) V(server) reports (-x)
- Report as a Comma-Separated Values (-y)
- Compatibility mode allows for use with older version of iPerf (-C)
iPerf 3 user documentation
| GENERAL OPTIONS | |
|---|---|
| Command line option | Description |
| -p, —port n | The server port for the server to listen on and the client to connect to. This should be the same in both client and server. Default is 5201. |
| —cport n | Option to specify the client-side port. (new in iPerf 3.1) |
| -f, —format [kmKM] | A letter specifying the format to print bandwidth numbers in. Supported formats are 'k' = Kbits/sec 'K' = KBytes/sec 'm' = Mbits/sec 'M' = MBytes/sec The adaptive formats choose between kilo- and mega- as appropriate. |
| -i, —interval n | Sets the interval time in seconds between periodic bandwidth, jitter, and loss reports. If non-zero, a report is made every interval seconds of the bandwidth since the last report. If zero, no periodic reports are printed. Default is zero. |
| -F, —file name | client-side: read from the file and write to the network, instead of using random data; server-side: read from the network and write to the file, instead of throwing the data away. |
| -A, —affinity n/n,m-F | Set the CPU affinity, if possible (Linux and FreeBSD only). On both the client and server you can set the local affinity by using the n form of this argument (where n is a CPU number). In addition, on the client side you can override the server’s affinity for just that one test, using the n,m form of argument. Note that when using this feature, a process will only be bound to a single CPU (as opposed to a set containing potentialy multiple CPUs). |
| -B, —bind host | Bind to host, one of this machine’s addresses. For the client this sets the outbound interface. For a server this sets the incoming interface. This is only useful on multihomed hosts, which have multiple network interfaces. |
| -V, —verbose | give more detailed output |
| -J, —json | output in JSON format |
| —logfile file | send output to a log file. (new in iPerf 3.1) |
| —d, —debug | emit debugging output. Primarily (perhaps exclusively) of use to developers. |
| -v, —version | Show version information and quit. |
| -h, —help | Show a help synopsis and quit. |
| SERVER SPECIFIC OPTIONS | |
| Command line option | Description |
| -s, —server | Run iPerf in server mode. (This will only allow one iperf connection at a time) |
| -D, —daemon | Run the server in background as a daemon. |
| -I, —pidfilefile | write a file with the process ID, most useful when running as a daemon. (new in iPerf 3.1) |
| CLIENT SPECIFIC OPTIONS | |
| Command line option | Description |
| -c, —client host | Run iPerf in client mode, connecting to an iPerf server running on host. |
| —sctp | Use SCTP rather than TCP (Linux, FreeBSD and Solaris). (new in iPerf 3.1) |
| -u, —udp | Use UDP rather than TCP. See also the -b option. |
| -b, —bandwidth n[KM] | Set target bandwidth to n bits/sec (default 1 Mbit/sec for UDP, unlimited for TCP). If there are multiple streams (-P flag), the bandwidth limit is applied separately to each stream. You can also add a ’/’ and a number to the bandwidth specifier. This is called «burst mode». It will send the given number of packets without pausing, even if that temporarily exceeds the specified bandwidth limit. |
| -t, —time n | The time in seconds to transmit for. iPerf normally works by repeatedly sending an array of len bytes for time seconds. Default is 10 seconds. See also the -l, -k and -n options. |
| -n, —num n[KM] | The number of buffers to transmit. Normally, iPerf sends for 10 seconds. The -n option overrides this and sends an array of len bytes num times, no matter how long that takes. See also the -l, -k and -t options. |
| -k, —blockcount n[KM] | The number of blocks (packets) to transmit. (instead of -t or -n) See also the -t, -l and -n options. |
| -l, —length n[KM] | The length of buffers to read or write. iPerf works by writing an array of len bytes a number of times. Default is 128 KB for TCP, 8 KB for UDP. See also the -n, -k and -t options. |
| -P, —parallel n | The number of simultaneous connections to make to the server. Default is 1. |
| -R, —reverse | Run in reverse mode (server sends, client receives). |
| -w, —window n[KM] | Sets the socket buffer sizes to the specified value. For TCP, this sets the TCP window size. (this gets sent to the server and used on that side too) |
| -M, —set-mss n | Attempt to set the TCP maximum segment size (MSS). The MSS is usually the MTU — 40 bytes for the TCP/IP header. For ethernet, the MSS is 1460 bytes (1500 byte MTU). |
| -N, —no-delay | Set the TCP no delay option, disabling Nagle’s algorithm. Normally this is only disabled for interactive applications like telnet. |
| -4, —version4 | only use IPv4. |
| -6, —version4 | only use IPv6. |
| -S, —tos n | The type-of-service for outgoing packets. (Many routers ignore the TOS field.) You may specify the value in hex with a ‘0x’ prefix, in octal with a ‘0’ prefix, or in decimal. For example, ‘0x10’ hex = ‘020’ octal = ’16’ decimal. The TOS numbers specified in RFC 1349 are: IPTOS_LOWDELAY minimize delay 0x10 IPTOS_THROUGHPUT maximize throughput 0x08 IPTOS_RELIABILITY maximize reliability 0x04 IPTOS_LOWCOST minimize cost 0x02 |
| -L, —flowlabel n | Set the IPv6 flow label (currently only supported on Linux). |
| -Z, —zerocopy | Use a «zero copy» method of sending data, such as sendfile(2), instead of the usual write(2). This uses much less CPU. |
| -O, —omit n | Omit the first n seconds of the test, to skip past the TCP TCP slowstart period. |
| -T, —title str | Prefix every output line with this string. |
| -C, —linux-congestion algo | Set the congestion control algorithm (Linux only for iPerf 3.0, Linux and FreeBSD for iPerf 3.1). |
See also https://github.com/esnet/iperf
Change between iPerf 2.0.6, iPerf 2.0.7 and iPerf 2.0.8
- 2.0.6 change set (rjmcmahon@rjmcmahon.com) March 2014 :
- Increase the shared memory for report headers reducing mutex contention. Needed to increase performance. Minor code change that should be platform/os independent
- 2.0.7 change set (rjmcmahon@rjmcmahon.com) August 2014 :
- Linux only version which supports end/end latency (assumes clocks synched)
- Support for smaller report interval (5 milliseconds or greater)
- End/end latency with UDP (mean/min/max), display in milliseconds with resolution of microseconds
- Socket read timeouts (server only) so iperf reports occur regardless of no received packets
- Report timestamps now display millisecond resolution
- Local bind supports port value using colon as delimeter (-B 10.10.10.1:60001)
- Use linux realtime scheduler and packet level timestamps for improved latency accuracy
- Suggest PTP on client and server to synch clocks to microsecond
- Suggest a quality reference for the PTP grandmaster such as a GPS disciplined oscillator from companies like Spectracom
- 2.0.8 change set (as of 12 january 2015) :
- Fix portability, compile and test with Linux, Win10, Win7, WinXP, MacOS and Android
- Client now requires -u for UDP (no longer defaults to UDP with -b)
- Maintain legacy report formats
- Support for -e to get enhanced reports
- Support TCP rate limited streams (via the -b) using token bucket
- Support packets per second (UDP) via pps as units, (e.g. -b 1000pps)
- Display PPS in both client and server reports (UDP)
- Support realtime scheduler as a command line option (—realtime or -z)
- Improve client tx code path so actual tx offerred rate will converge to the -b value
- Improve accuracy of microsecond delay calls (in platform independent manner)
- (Use of Kalman filter to predict delay errors and adjust delays per predicted error)
- Display target loop time in initial client header (UDP)
- Fix final latency report sent from server to client (UDP)
- Include standard deviation in latency output
- Suppress unrealistic latency output (-/-/-/-)
- Support SO_SNDTIMEO on send so socket write won’t block beyond -t (TCP)
- Use clock_gettime if available (preferred over gettimeofday())
- TCP write and error counts (TCP retries and CWND for linux)
- TCP read count, TCP read histogram (8 bins)
- Server will close the socket after -t seconds of no traffic
See also https://sourceforge.net/projects/iperf2/
iPerf 2 user documentation
| GENERAL OPTIONS | ||
|---|---|---|
| Command line option | Environment variable option | Description |
| -f, —format [bkmaBKMA] | $IPERF_FORMAT | A letter specifying the format to print bandwidth numbers in. Supported formats are 'b' = bits/sec 'B' = Bytes/sec 'k' = Kbits/sec 'K' = KBytes/sec 'm' = Mbits/sec 'M' = MBytes/sec 'g' = Gbits/sec 'G' = GBytes/sec 'a' = adaptive bits/sec 'A' = adaptive Bytes/sec The adaptive formats choose between kilo- and mega- as appropriate. Fields |
| -i, —interval # | $IPERF_INTERVAL | Sets the interval time in seconds between periodic bandwidth, jitter, and loss reports. If non-zero, a report is made every interval seconds of the bandwidth since the last report. If zero, no periodic reports are printed. Default is zero. |
| -l, —len #[KM] | $IPERF_LEN | The length of buffers to read or write. iPerf works by writing an array of len bytes a number of times. Default is 8 KB for TCP, 1470 bytes for UDP. Note for UDP, this is the datagram size and needs to be lowered when using IPv6 addressing to 1450 or less to avoid fragmentation. See also the -n and -t options. |
| -m, —print_mss | $IPERF_PRINT_MSS | Print the reported TCP MSS size (via the TCP_MAXSEG option) and the observed read sizes which often correlate with the MSS. The MSS is usually the MTU — 40 bytes for the TCP/IP header. Often a slightly smaller MSS is reported because of extra header space from IP options. The interface type corresponding to the MTU is also printed (ethernet, FDDI, etc.). This option is not implemented on many OSes, but the read sizes may still indicate the MSS. |
| -p, —port # | $IPERF_PORT | The server port for the server to listen on and the client to connect to. This should be the same in both client and server. Default is 5001, the same as ttcp. |
| -u, —udp | $IPERF_UDP | Use UDP rather than TCP. See also the -b option. |
| -w, —window #[KM] | $TCP_WINDOW_SIZE | Sets the socket buffer sizes to the specified value. For TCP, this sets the TCP window size. For UDP it is just the buffer which datagrams are received in, and so limits the largest receivable datagram size. |
| -B, —bind host | $IPERF_BIND | Bind to host, one of this machine’s addresses. For the client this sets the outbound interface. For a server this sets the incoming interface. This is only useful on multihomed hosts, which have multiple network interfaces. For iPerf in UDP server mode, this is also used to bind and join to a |
| -C, —compatibility | $IPERF_COMPAT | Compatibility mode allows for use with older version of iPerf. This mode is not required for interoperability but it is highly recommended. In some cases when using representative streaming you could cause a 1.7 server to crash or cause undesired connection attempts. |
| -M, —mss #[KM} | $IPERF_MSS | Attempt to set the TCP maximum segment size (MSS) via the TCP_MAXSEG option. The MSS is usually the MTU — 40 bytes for the TCP/IP header. For ethernet, the MSS is 1460 bytes (1500 byte MTU). This option is not implemented on many OSes. |
| -N, —nodelay | $IPERF_NODELAY | Set the TCP no delay option, disabling Nagle’s algorithm. Normally this is only disabled for interactive applications like telnet. |
| -V (from v1.6 or higher) | . | Bind to an IPv6 address Server side: $ iperf -s -V Client side: Note: On version 1.6.3 and later a specific IPv6 Address does |
| -h, —help | Print out a summary of commands and quit. | |
| -v, —version | Print version information and quit. Prints ‘pthreads’ if compiled with POSIX threads, ‘win32 threads’ if compiled with Microsoft Win32 threads, or ‘single threaded’ if compiled without threads. |
|
| SERVER SPECIFIC OPTIONS | ||
| Command line option | Environment variable option | Description |
| -s, —server | $IPERF_SERVER | Run iPerf in server mode. (iPerf2 can handle multiple client requests) |
| -D (from v1.2 or higher) | . | Run the server as a daemon (Unix platforms) On Win32 platforms where services are available, iPerf will start running as a service. |
| -R (only for Windows, from v1.2 or higher) | . | Remove the iPerf service (if it’s running). |
| -o (only for Windows, from v1.2 or higher) | . | Redirect output to given file. |
| -c, —client host | $IPERF_CLIENT | If iPerf is in server mode, then specifying a host with -c will limit the connections that iPerf will accept to the host specified. Does not work well for UDP. |
| -P, —parallel # | $IPERF_PARALLEL | The number of connections to handle by the server before closing. Default is 0 (which means to accept connections forever). |
| CLIENT SPECIFIC OPTIONS | ||
| Command line option | Environment variable option | Description |
| -b, —bandwidth #[KM] | $IPERF_BANDWIDTH | The UDP bandwidth to send at, in bits/sec. This implies the -u option. Default is 1 Mbit/sec. |
| -c, —client host | $IPERF_CLIENT | Run iPerf in client mode, connecting to an iPerf server running on host. |
| -d, —dualtest | $IPERF_DUALTEST | Run iPerf in dual testing mode. This will cause the server to connect back to the client on the port specified in the -L option (or defaults to the port the client connected to the server on). This is done immediately therefore running the tests simultaneously. If you want an alternating test try -r. |
| -n, —num #[KM] | $IPERF_NUM | The number of buffers to transmit. Normally, iPerf sends for 10 seconds. The -n option overrides this and sends an array of len bytes num times, no matter how long that takes. See also the -l and -t options. |
| -r, —tradeoff | $IPERF_TRADEOFF | Run iPerf in tradeoff testing mode. This will cause the server to connect back to the client on the port specified in the -L option (or defaults to the port the client connected to the server on). This is done following the client connection termination, therefore running the tests alternating. If you want an simultaneous test try -d. |
| -t, —time # | $IPERF_TIME | The time in seconds to transmit for. iPerf normally works by repeatedly sending an array of len bytes for time seconds. Default is 10 seconds. See also the -l and -n options. |
| -L, —listenport # | $IPERF_LISTENPORT | This specifies the port that the server will connect back to the client on. It defaults to the port used to connect to the server from the client. |
| -P, —parallel # | $IPERF_PARALLEL | The number of simultaneous connections to make to the server. Default is 1. Requires thread support on both the client and server. |
| -S, —tos # | $IPERF_TOS | The type-of-service for outgoing packets. (Many routers ignore the TOS field.) You may specify the value in hex with a ‘0x’ prefix, in octal with a ‘0’ prefix, or in decimal. For example, ‘0x10’ hex = ‘020’ octal = ’16’ decimal. The TOS numbers specified in RFC 1349 are: IPTOS_LOWDELAY minimize delay 0x10 IPTOS_THROUGHPUT maximize throughput 0x08 IPTOS_RELIABILITY maximize reliability 0x04 IPTOS_LOWCOST minimize cost 0x02 |
| -T, —ttl # | $IPERF_TTL | The time-to-live for outgoing multicast packets. This is essentially the number of router hops to go through, and is also used for scoping. Default is 1, link-local. |
| -F (from v1.2 or higher) | . | Use a representative stream to measure bandwidth, e.g. :- $ iperf -c <server address> -F <file-name> |
| -I (from v1.2 or higher) | . | Same as -F, input from stdin. |
Tuning a TCP connection
The primary goal of iPerf is to help in tuning TCP connections over a particular path. The most
fundamental tuning issue for TCP is the TCP window size, which controls how much data can be in the network at any one point.
If it is too small, the sender will be idle at times and get poor performance. The theoretical value to use for the
TCP window size is the bandwidth delay product,
bottleneck bandwidth * round trip time
In the below modi4/cyclops example, the bottleneck link is a 45 Mbit/sec DS3 link and the round trip time measured with ping is 42 ms.
The bandwidth delay product is
45 Mbit/sec * 42 ms
= (45e6) * (42e-3)
= 1890000 bits
= 230 KByte
That is a starting point for figuring the best window size; setting it higher or lower may produce better results.
In our example, buffer sizes over 130K did not improve the performance, despite the bandwidth delay product of 230K.
Note that many OSes and hosts have upper limits on the TCP window size.
These may be as low as 64 KB, or as high as several MB. iPerf tries to detect when these occur and give a warning that the actual and requested window sizes are
not equal (as below, though that is due to rounding in IRIX).
For more information on TCP window sizes, see the LaFibre.info.
Here is an example session, between node1 in Illinois and node2 in North Carolina. These are connected via the vBNS backbone and a 45 Mbit/sec DS3 link.
Notice we improve bandwidth performance by a factor of 3 using proper TCP window sizes.
Use the adaptive window sizes feature on platforms which allow setting window sizes in the granularity of bytes.
node2> iperf -s ------------------------------------------------------------ Server listening on TCP port 5001 TCP window size: 60.0 KByte (default) ------------------------------------------------------------ [ 4] local <IP Addr node2> port 5001 connected with <IP Addr node1> port 2357 [ ID] Interval Transfer Bandwidth [ 4] 0.0-10.1 sec 6.5 MBytes 5.2 Mbits/sec node1> iperf -c node2 ------------------------------------------------------------ Client connecting to node1, TCP port 5001 TCP window size: 59.9 KByte (default) ------------------------------------------------------------ [ 3] local <IP Addr node1> port 2357 connected with <IP Addr node2> port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 6.5 MBytes 5.2 Mbits/sec
node2> iperf -s -w 130k ------------------------------------------------------------ Server listening on TCP port 5001 TCP window size: 130 KByte ------------------------------------------------------------ [ 4] local <IP Addr node 2> port 5001 connected with <IP Addr node 1> port 2530 [ ID] Interval Transfer Bandwidth [ 4] 0.0-10.1 sec 19.7 MBytes 15.7 Mbits/sec node1> iperf -c node2 -w 130k ------------------------------------------------------------ Client connecting to node2, TCP port 5001 TCP window size: 129 KByte (WARNING: requested 130 KByte) ------------------------------------------------------------ [ 3] local <IP Addr node1> port 2530 connected with <IP Addr node2> port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 19.7 MBytes 15.8 Mbits/sec
Another test to do is run parallel TCP streams.
If the total aggregate bandwidth is more than what an individual stream gets, something is wrong.
Either the TCP window size is too small, or the OS’s TCP implementation has bugs, or the network itself has deficiencies.
See above for TCP window sizes; otherwise diagnosing which is somewhat difficult.
If iPerf is compiled with pthreads, a single client and server can test this, otherwise setup multiple clients and servers on different ports.
Here’s an example where a single stream gets 16.5 Mbit/sec, but two parallel streams
together get 16.7 + 9.4 = 26.1 Mbit/sec, even when using large TCP window sizes:
node2> iperf -s -w 300k ------------------------------------------------------------ Server listening on TCP port 5001 TCP window size: 300 KByte ------------------------------------------------------------ [ 4] local <IP Addr node2> port 5001 connected with <IP Addr node1> port 6902 [ ID] Interval Transfer Bandwidth [ 4] 0.0-10.2 sec 20.9 MBytes 16.5 Mbits/sec [ 4] local <IP Addr node2> port 5001 connected with <IP Addr node1> port 6911 [ 5] local <IP Addr node2> port 5001 connected with <IP Addr node2> port 6912 [ ID] Interval Transfer Bandwidth [ 5] 0.0-10.1 sec 21.0 MBytes 16.7 Mbits/sec [ 4] 0.0-10.3 sec 12.0 MBytes 9.4 Mbits/sec node1> ./iperf -c node2 -w 300k ------------------------------------------------------------ Client connecting to node2, TCP port 5001 TCP window size: 299 KByte (WARNING: requested 300 KByte) ------------------------------------------------------------ [ 3] local <IP Addr node2> port 6902 connected with <IP Addr node1> port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.2 sec 20.9 MBytes 16.4 Mbits/sec node1> iperf -c node2 -w 300k -P 2 ------------------------------------------------------------ Client connecting to node2, TCP port 5001 TCP window size: 299 KByte (WARNING: requested 300 KByte) ------------------------------------------------------------ [ 4] local <IP Addr node2> port 6912 connected with <IP Addr node1> port 5001 [ 3] local <IP Addr node2> port 6911 connected with <IP Addr node1> port 5001 [ ID] Interval Transfer Bandwidth [ 4] 0.0-10.1 sec 21.0 MBytes 16.6 Mbits/sec [ 3] 0.0-10.2 sec 12.0 MBytes 9.4 Mbits/sec
A secondary tuning issue for TCP is the maximum transmission unit (MTU).
To be most effective, both hosts should support Path MTU Discovery.
Hosts without Path MTU Discovery often use 536 as the MSS, which wastes bandwidth and processing time.
Use the -m option to display what MSS is being used, and see if this matches what you expect.
Often it is around 1460 bytes for ethernet.
node3> iperf -s -m
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 60.0 KByte (default)
------------------------------------------------------------
[ 4] local <IP Addr node3> port 5001 connected with <IP Addr node4> port 1096
[ ID] Interval Transfer Bandwidth
[ 4] 0.0- 2.0 sec 1.8 MBytes 6.9 Mbits/sec
[ 4] MSS size 1448 bytes (MTU 1500 bytes, ethernet)
[ 4] Read lengths occurring in more than 5% of reads:
[ 4] 952 bytes read 219 times (16.2%)
[ 4] 1448 bytes read 1128 times (83.6%)
Here is a host that doesn’t support Path MTU Discovery. It will only send and receive small 576 byte packets.
node4> iperf -s -m ------------------------------------------------------------ Server listening on TCP port 5001 TCP window size: 32.0 KByte (default) ------------------------------------------------------------ [ 4] local <IP Addr node4> port 5001 connected with <IP Addr node3> port 13914 [ ID] Interval Transfer Bandwidth [ 4] 0.0- 2.3 sec 632 KBytes 2.1 Mbits/sec WARNING: Path MTU Discovery may not be enabled. [ 4] MSS size 536 bytes (MTU 576 bytes, minimum) [ 4] Read lengths occurring in more than 5% of reads: [ 4] 536 bytes read 308 times (58.4%) [ 4] 1072 bytes read 91 times (17.3%) [ 4] 1608 bytes read 29 times (5.5%)
iPerf supports other tuning options, which were added for exceptional network situations like HIPPI-to-HIPPI over ATM.
Tuning a UDP connection
iPerf creates a constant bit rate UDP stream. This is a very artificial stream, similar to voice communication but not much else.
You will want to adjust the datagram size (-l) to the size your application uses.
The server detects UDP datagram loss by ID numbers in the datagrams.
Usually a UDP datagram becomes several IP packets. Losing a single IP packet will lose the entire datagram.
To measure packet loss instead of datagram loss, make the datagrams small enough to fit into a single packet, using the -l option.
The default size of 1470 bytes works for ethernet. Out-of-order packets are also detected.
(Out-of-order packets cause some ambiguity in the lost packet count;
iPerf assumes they are not duplicate packets, so they are excluded from the lost packet count.)
Since TCP does not report loss to the user, I find UDP tests helpful to see packet loss along a path.
Jitter calculations are continuously computed by the server, as specified by
RTP in RFC 1889. The client records a 64 bit second/microsecond timestamp in the
packet. The server computes the relative transit time as (server’s receive time
— client’s send time). The client’s and server’s clocks do not need to be
synchronized; any difference is subtracted out in the jitter calculation. Jitter
is the smoothed mean of differences between consecutive transit times.
node2> iperf -s -u -i 1 ------------------------------------------------------------ Server listening on UDP port 5001 Receiving 1470 byte datagrams UDP buffer size: 60.0 KByte (default) ------------------------------------------------------------ [ 4] local <IP Addr node2> port 5001 connected with <IP Addr node1> port 9726 [ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams [ 4] 0.0- 1.0 sec 1.3 MBytes 10.0 Mbits/sec 0.209 ms 1/ 894 (0.11%) [ 4] 1.0- 2.0 sec 1.3 MBytes 10.0 Mbits/sec 0.221 ms 0/ 892 (0%) [ 4] 2.0- 3.0 sec 1.3 MBytes 10.0 Mbits/sec 0.277 ms 0/ 892 (0%) [ 4] 3.0- 4.0 sec 1.3 MBytes 10.0 Mbits/sec 0.359 ms 0/ 893 (0%) [ 4] 4.0- 5.0 sec 1.3 MBytes 10.0 Mbits/sec 0.251 ms 0/ 892 (0%) [ 4] 5.0- 6.0 sec 1.3 MBytes 10.0 Mbits/sec 0.215 ms 0/ 892 (0%) [ 4] 6.0- 7.0 sec 1.3 MBytes 10.0 Mbits/sec 0.325 ms 0/ 892 (0%) [ 4] 7.0- 8.0 sec 1.3 MBytes 10.0 Mbits/sec 0.254 ms 0/ 892 (0%) [ 4] 8.0- 9.0 sec 1.3 MBytes 10.0 Mbits/sec 0.282 ms 0/ 892 (0%) [ 4] 0.0-10.0 sec 12.5 MBytes 10.0 Mbits/sec 0.243 ms 1/ 8922 (0.011%) node1> iperf -c node2 -u -b 10m ------------------------------------------------------------ Client connecting to node2, UDP port 5001 Sending 1470 byte datagrams UDP buffer size: 60.0 KByte (default) ------------------------------------------------------------ [ 3] local <IP Addr node1> port 9726 connected with <IP Addr node2> port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 12.5 MBytes 10.0 Mbits/sec [ 3] Sent 8922 datagrams
Notice the higher jitter due to datagram reassembly when using larger 32 KB datagrams, each split into 23 packets of 1500 bytes.
The higher datagram loss seen here may be due to the burstiness of the traffic, which is 23 back-to-back packets and then a long
pause, rather than evenly spaced individual packets.
node2> iperf -s -u -l 32k -w 128k -i 1 ------------------------------------------------------------ Server listening on UDP port 5001 Receiving 32768 byte datagrams UDP buffer size: 128 KByte ------------------------------------------------------------ [ 3] local <IP Addr node2> port 5001 connected with <IP Addr node1> port 11303 [ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams [ 3] 0.0- 1.0 sec 1.3 MBytes 10.0 Mbits/sec 0.430 ms 0/ 41 (0%) [ 3] 1.0- 2.0 sec 1.1 MBytes 8.5 Mbits/sec 5.996 ms 6/ 40 (15%) [ 3] 2.0- 3.0 sec 1.2 MBytes 9.7 Mbits/sec 0.796 ms 1/ 40 (2.5%) [ 3] 3.0- 4.0 sec 1.2 MBytes 10.0 Mbits/sec 0.403 ms 0/ 40 (0%) [ 3] 4.0- 5.0 sec 1.2 MBytes 10.0 Mbits/sec 0.448 ms 0/ 40 (0%) [ 3] 5.0- 6.0 sec 1.2 MBytes 10.0 Mbits/sec 0.464 ms 0/ 40 (0%) [ 3] 6.0- 7.0 sec 1.2 MBytes 10.0 Mbits/sec 0.442 ms 0/ 40 (0%) [ 3] 7.0- 8.0 sec 1.2 MBytes 10.0 Mbits/sec 0.342 ms 0/ 40 (0%) [ 3] 8.0- 9.0 sec 1.2 MBytes 10.0 Mbits/sec 0.431 ms 0/ 40 (0%) [ 3] 9.0-10.0 sec 1.2 MBytes 10.0 Mbits/sec 0.407 ms 0/ 40 (0%) [ 3] 0.0-10.0 sec 12.3 MBytes 9.8 Mbits/sec 0.407 ms 7/ 401 (1.7%) node1> iperf -c node2 -b 10m -l 32k -w 128k ------------------------------------------------------------ Client connecting to node2, UDP port 5001 Sending 32768 byte datagrams UDP buffer size: 128 KByte ------------------------------------------------------------ [ 3] local <IP Addr node2> port 11303 connected with <IP Addr node1> port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 12.5 MBytes 10.0 Mbits/sec [ 3] Sent 401 datagrams
Multicast
To test multicast, run several servers with the bind option (-B, —bind) set
to the multicast group address. Run the client, connecting to the multicast
group address and setting the TTL (-T, —ttl) as needed. Unlike normal TCP and
UDP tests, multicast servers may be started after the client. In that case,
datagrams sent before the server started show up as losses in the first periodic
report (61 datagrams on arno below).
node5> iperf -c 224.0.67.67 -u --ttl 5 -t 5 ------------------------------------------------------------ Client connecting to 224.0.67.67, UDP port 5001 Sending 1470 byte datagrams Setting multicast TTL to 5 UDP buffer size: 32.0 KByte (default) ------------------------------------------------------------ [ 3] local <IP Addr node5> port 1025 connected with 224.0.67.67 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0- 5.0 sec 642 KBytes 1.0 Mbits/sec [ 3] Sent 447 datagrams node5> iperf -s -u -B 224.0.67.67 -i 1 ------------------------------------------------------------ Server listening on UDP port 5001 Binding to local address 224.0.67.67 Joining multicast group 224.0.67.67 Receiving 1470 byte datagrams UDP buffer size: 32.0 KByte (default) ------------------------------------------------------------ [ 3] local 224.0.67.67 port 5001 connected with <IP Addr node5> port 1025 [ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams [ 3] 0.0- 1.0 sec 131 KBytes 1.0 Mbits/sec 0.007 ms 0/ 91 (0%) [ 3] 1.0- 2.0 sec 128 KBytes 1.0 Mbits/sec 0.008 ms 0/ 89 (0%) [ 3] 2.0- 3.0 sec 128 KBytes 1.0 Mbits/sec 0.010 ms 0/ 89 (0%) [ 3] 3.0- 4.0 sec 128 KBytes 1.0 Mbits/sec 0.013 ms 0/ 89 (0%) [ 3] 4.0- 5.0 sec 128 KBytes 1.0 Mbits/sec 0.008 ms 0/ 89 (0%) [ 3] 0.0- 5.0 sec 642 KBytes 1.0 Mbits/sec 0.008 ms 0/ 447 (0%) node6> iperf -s -u -B 224.0.67.67 -i 1 ------------------------------------------------------------ Server listening on UDP port 5001 Binding to local address 224.0.67.67 Joining multicast group 224.0.67.67 Receiving 1470 byte datagrams UDP buffer size: 60.0 KByte (default) ------------------------------------------------------------ [ 3] local 224.0.67.67 port 5001 connected with <IP Addr node5> port 1025 [ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams [ 3] 0.0- 1.0 sec 129 KBytes 1.0 Mbits/sec 0.778 ms 61/ 151 (40%) [ 3] 1.0- 2.0 sec 128 KBytes 1.0 Mbits/sec 0.236 ms 0/ 89 (0%) [ 3] 2.0- 3.0 sec 128 KBytes 1.0 Mbits/sec 0.264 ms 0/ 89 (0%) [ 3] 3.0- 4.0 sec 128 KBytes 1.0 Mbits/sec 0.248 ms 0/ 89 (0%) [ 3] 0.0- 4.3 sec 554 KBytes 1.0 Mbits/sec 0.298 ms 61/ 447 (14%)
Start multiple clients or servers as explained above, sending data to the same multicast server.
(If you have multiple servers listening on the multicast address, each of the servers will be getting the data)
IPv6 Mode
- Get the IPv6 address of the node using the ‘ifconfig’ command.
Use the -V option to indicate that you are using an IPv6 address Please note that we need to explicitly bind the server address also.Server side:
$ iperf -s -VClient side:
$ iperf -c <Server IPv6 Address> -V>Note: iPerf version 1.6.2 and eariler require a IPv6 address to be explicitly bound
with the -B option for the server.
Using Representative Streams to measure bandwidth
- Use the -F or -I option. If you want to test how your network performs
with compressed / uncompressed streams, just create representative streams and
use the -F option to test it. This is usually due to the link layer
compressing data.The -F option is for file input.
The -I option is for input from stdin.E.g.
Client: $ iperf -c <server address> -F <file-name>Client: $ iperf -c <server address> -I
Running the server as a daemon
- Use the -D command line option to run the server as a daemon. Redirect the
output to a file.
E.g. iperf -s -D >
iperflog. This will have the iPerf Server running
as a daemon and the server messages will be logged in the file iperfLog.
Using iPerf as a Service under Win32
- There are three options for Win32:
- -o outputfilename
- output the messages into the specified file
- -s -D
- install iPerf as a service and run it
- -s -R
- uninstall the iPerf service
Examples:
- iperf -s -D -o iperflog.txt
- will install the iPerf service and run it. Messages will be reported into «%windir%\system32\iperflog.txt»
- iperf -s -R
- will uninstall the iPerf service if it is installed.
Note: If you stop want to restart the iPerf service after having killed it with the Microsoft
Management Console or the Windows Task Manager, make sure to use the proper OPTION in the service properties dialog.
Adaptive window sizes (under development)
- Use the -W option on the client to run the client with the adaptive window size.
Ensure that the server window size is fairly big for this option.
E.g.. If the server TCP window size is 8KB, it does not help having a client TCP window size of 256KB.
256KB Server TCP Window Size should suffice for most high bandwidth networks.Client changes the TCP window size using a binary exponential algorithm.
This means that you may notice that TCP window size suggested may vary according to the traffic in the network,
iPerf will suggest the best window size for the current network scenario.
Compiling
Once you have the distribution, on UNIX, unpack it using gzip and tar.
That will create a new directory ‘iperf-<version#>’ with the source files and documentation.
iPerf compiles cleanly on many systems including Linux, SGI IRIX, HP-UX,
Solaris, AIX, and Cray UNICOS. Use ‘make‘ to configure for your OS and compile the source code.
gunzip -c iperf-<version>.tar.gz | tar -xvf - cd iperf-<version> ./configure make
To install iPerf, use ‘make install‘, which will ask you where to install it.
To recompile, the easiest way is to start over. Do ‘make distclean‘ then ‘./configure; make‘.
See the Makefile for more options.
