Measuring network usage and identifying established connections on a Debian Linux server.
Install Performance Monitoring Tools
Install sysstat, vnstat and iftop, if not yet installed:
# apt-get update && apt-get install sysstat vnstat iftop
For RHEL/CentOS, do the following (must have EPEL repository enabled):
# yum install -y sysstat vnstat iftop
The sysstat package contains sar and iostat system performance tools which we’ll be using today.
Make sure that sar is enabled in /etc/default/sysstat. If not enabled, do it.
You may also want to change the history value in /etc/sysstat/sysstat to something different than 7 days:
HISTORY=60
Note that if value is greater than 28, then log files will be kept in multiple directories, one for each month.
By default sysstat will collect data every 10 minutes. You can change this by modifying the cronjob /etc/cron.d/sysstat.
Finally, restart the service:
# service sysstat restart
Measure Network Usage
Network Usage with netstat
The netstat command prints information about the Linux networking subsystem. By default, netstat displays a list of open sockets.
Different arguments can be passed to print the desired type of information. Some of available arguments are listed below:
- -a: show both listening and non-listening sockets.
- -r: display the kernel routing table. Note that netstat -r and route -e produce the same output. However, route -e command requires root privileges to be run where netstat -r doesn’t.
- -i: display a table of all network interfaces.
- -e: display additional information.
- -p: show the PID and name of the program to which each socket belongs.
- -l: show listening sockets only.
- -t: show TCP connections.
- -u: show UDP connections.
- -c: print selected information every second continuously.
- -M: display a list of masqueraded connections (only works on pre-2.4 kernels). The ipt_MASQUERADE module has to be loaded.
On new kernels (3.2 f.e.), the following message appears:
$ netstat -M netstat: no support for `ip_masquerade' on this system.
It is advised to use the /proc/net/ip_conntrack on new kernels to get a list of masqueraded connections.
Show all established TCP (-t) connections by using numerical ports and addresses (-n):
$ netstat -nt | grep -i established tcp 0 0 10.10.1.17:12 10.10.1.10:56839 ESTABLISHED tcp 0 0 10.10.1.17:53669 10.10.1.8:10050 ESTABLISHED tcp6 0 0 10.10.1.17:443 10.10.1.10:42590 ESTABLISHED tcp6 0 0 10.10.1.17:443 10.10.1.10:42606 ESTABLISHED tcp6 0 0 10.10.1.17:443 10.10.1.10:42605 ESTABLISHED
Preview network statistics (-s) for all protocols:
$ netstat -s | less
Network Usage with vnstat
The vnstat is a console-based network traffic monitor. It keeps a log of hourly, daily and monthly network traffic for the selected interfaces.
Show network usage for the eth0 interface (-i):
$ vnstat -i eth0
Database updated: Sat Mar 1 17:16:49 2014
eth0 since 02/17/14
rx: 1.87 GiB tx: 1.58 GiB total: 3.45 GiB
monthly
rx | tx | total | avg. rate
------------------------+-------------+-------------+---------------
Feb '14 1.61 GiB | 1.36 GiB | 2.97 GiB | 10.30 kbit/s
Mar '14 129.66 MiB | 119.31 MiB | 248.97 MiB | 32.79 kbit/s
------------------------+-------------+-------------+---------------
estimated 5.42 GiB | 5.00 GiB | 10.43 GiB |
daily
rx | tx | total | avg. rate
------------------------+-------------+-------------+---------------
yesterday 156.53 MiB | 135.77 MiB | 292.30 MiB | 27.71 kbit/s
today 129.66 MiB | 119.31 MiB | 248.97 MiB | 32.79 kbit/s
------------------------+-------------+-------------+---------------
estimated 179 MiB | 165 MiB | 344 MiB |
Select the eth0 interface (-i) and display traffic by using the short mode (-s):
$ vnstat -i eth0 -s
rx / tx / total / estimated
eth0:
Feb '14 1.61 GiB / 1.36 GiB / 2.97 GiB
Mar '14 129.66 MiB / 119.31 MiB / 248.97 MiB / 10.43 GiB
yesterday 156.53 MiB / 135.77 MiB / 292.30 MiB
today 129.66 MiB / 119.31 MiB / 248.97 MiB / 344 MiB
Display current transfer rate for the eth0 interface in real time until interrupted. Statistics will be shown after interruption if the runtime was more than 10 seconds:
# vnstat -i eth0 -l
Monitoring eth0... (press CTRL-C to stop)
rx: 0 kbit/s 0 p/s tx: 0 kbit/s 0 p/s^C
eth0 / traffic statistics
rx | tx
--------------------------------------+------------------
bytes 176.99 MiB | 704.42 MiB
--------------------------------------+------------------
max 242.34 Mbit/s | 251.37 Mbit/s
average 16.29 Mbit/s | 64.84 Mbit/s
min 0 kbit/s | 0 kbit/s
--------------------------------------+------------------
packets 325642 | 123123
--------------------------------------+------------------
max 20681 p/s | 5966 p/s
average 3658 p/s | 1383 p/s
min 0 p/s | 0 p/s
--------------------------------------+------------------
time 1.48 minutes
Select the eth0 interface (-i) and show traffic for days (-d):
$ vnstat -i eth0 -d
eth0 / daily
day rx | tx | total | avg. rate
------------------------+-------------+-------------+---------------
02/17/14 43.75 MiB | 16.66 MiB | 60.41 MiB | 5.73 kbit/s
02/18/14 94.95 MiB | 88.40 MiB | 183.35 MiB | 17.38 kbit/s
12/20/14 0 KiB | 0 KiB | 0 KiB | 0.00 kbit/s
02/18/14 24.16 MiB | 16.39 MiB | 40.55 MiB | 3.84 kbit/s
02/19/14 147.66 MiB | 131.22 MiB | 278.89 MiB | 26.44 kbit/s
02/20/14 161.49 MiB | 134.28 MiB | 295.76 MiB | 28.04 kbit/s
02/21/14 169.64 MiB | 137.29 MiB | 306.93 MiB | 29.10 kbit/s
02/22/14 166.96 MiB | 142.06 MiB | 309.02 MiB | 29.30 kbit/s
02/23/14 166.68 MiB | 139.80 MiB | 306.48 MiB | 29.06 kbit/s
02/24/14 170.39 MiB | 144.82 MiB | 315.21 MiB | 29.89 kbit/s
02/25/14 157.14 MiB | 137.00 MiB | 294.14 MiB | 27.89 kbit/s
02/26/14 168.33 MiB | 138.85 MiB | 307.18 MiB | 29.13 kbit/s
02/27/14 160.50 MiB | 136.23 MiB | 296.73 MiB | 28.13 kbit/s
02/28/14 156.53 MiB | 135.77 MiB | 292.30 MiB | 27.71 kbit/s
03/01/14 129.66 MiB | 119.31 MiB | 248.97 MiB | 32.79 kbit/s
------------------------+-------------+-------------+---------------
estimated 179 MiB | 165 MiB | 344 MiB |
Network Usage with sar
Report network (-n) statistics from the network devices (DEV):
$ sar -n DEV 1 2 Linux 3.2.0-4-686-pae (flames) 01/03/14 _i686_ (2 CPU) 17:25:57 IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s 17:25:58 lo 14.14 14.14 0.76 0.76 0.00 0.00 0.00 17:25:58 eth0 52.53 65.66 0.00 0.00 0.00 0.00 0.00 17:25:58 IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s 17:25:59 lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 17:25:59 eth0 351.00 336.00 0.00 0.00 0.00 0.00 0.00 Average: IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s Average: lo 7.04 7.04 0.38 0.38 0.00 0.00 0.00 Average: eth0 202.51 201.51 0.00 0.00 0.00 0.00 0.00
The following values are displayed (as per man page):
- IFACE: name of the network interface for which statistics are reported.
- rxpck/s: total number of packets received per second.
- txpck/s: total number of packets transmitted per second.
- rxkB/s: total number of kilobytes received per second.
- txkB/s: total number of kilobytes transmitted per second.
- rxcmp/s: number of compressed packets received per second.
- txcmp/s: number of compressed packets transmitted per second.
- rxmcst/s: number of multicast packets received per second.
Report historical network (-n) statistics from the network devices (DEV) starting (-s) 1 PM and ending (-e) 2 PM time interval:
$ sar -n DEV -s 13:00:00 -e 14:00:00 Linux 3.2.0-4-686-pae (flames) 01/03/14 _i686_ (2 CPU) 13:05:01 IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s 13:15:01 lo 22.17 22.17 6.04 6.04 0.00 0.00 0.00 13:15:01 eth0 144.48 149.90 0.00 0.00 0.00 0.00 0.00 13:25:01 lo 27.09 27.09 34.76 34.76 0.00 0.00 0.00 13:25:01 eth0 159.49 157.17 0.00 0.00 0.00 0.00 0.00 13:35:01 lo 21.91 21.91 6.13 6.13 0.00 0.00 0.00 13:35:01 eth0 141.16 146.25 0.00 0.00 0.00 0.00 0.00 13:45:01 lo 20.91 20.91 5.85 5.85 0.00 0.00 0.00 13:45:01 eth0 137.24 143.68 0.00 0.00 0.00 0.00 0.00 13:55:01 lo 26.50 26.50 39.18 39.18 0.00 0.00 0.00 13:55:01 eth0 148.30 151.32 0.00 0.00 0.00 0.00 0.00 Average: lo 23.71 23.71 18.39 18.39 0.00 0.00 0.00 Average: eth0 146.13 149.66 0.00 0.00 0.00 0.00 0.00
For ultimate sar experience, try the following command:
$ sar -n ALL 1 1
Network Usage with iftop
The iftop command displays network bandwidth usage for the interface specified or for the first external interface it can find. Iftop must be run with sufficient permissions (usually as root) to monitor all network traffic.
Get all TCP connections with sent and received traffic on separate lines:
# iftop -nNPb -i eth0 -f tcp
-----------------------------------------------------------------------------------
10.10.1.17:12 => 10.10.1.10:56536 3.78kb 4.03kb 3.94kb
<= 416b 493b 316b
10.10.1.17:51909 => 10.10.1.8:10050 0b 233b 61b
<= 0b 232b 61b
10.10.1.17:51918 => 10.10.1.8:10050 0b 234b 61b
<= 0b 231b 61b
10.10.1.17:51921 => 10.10.1.8:10050 1.14kb 233b 61b
<= 1.13kb 232b 61b
10.10.1.17:51902 => 10.10.1.8:10050 0b 232b 61b
<= 0b 232b 61b
10.10.1.17:51912 => 10.10.1.8:10050 0b 232b 61b
<= 0b 232b 61b
10.10.1.17:51915 => 10.10.1.8:10050 0b 231b 61b
<= 0b 231b 61b
10.10.1.17:443 => 10.10.1.10:42515 0b 42b 76.2kb
<= 0b 42b 1.74kb
10.10.1.17:443 => 10.10.1.10:42509 0b 42b 1.94kb
<= 0b 42b 1.39kb
10.10.1.17:443 => 10.10.1.10:42527 0b 42b 433b
<= 0b 42b 495b
-----------------------------------------------------------------------------------
TX: cum: 642kB peak: 1.62Mb rates: 4.92kb 5.63kb 135kb
RX: 200kB 288kb 1.54kb 2.08kb 42.1kb
TOTAL: 842kB 1.75Mb 6.46kb 7.72kb 177kb
Parameters used are below:
- -n: don’t do hostname lookups..
- -N: don’t resolve port number to service names.
- -P: turn on port display.
- -b: don’t display bar graphs of traffic.
- -i: listen to packets on the interface specified.
- -f: use filter code to select the IP packets to count.
Get total traffic shown for SSH port 12 (note that we use a custom SSH port here, the default is TCP 22):
# iftop -nNPb -i eth0 -f "dst port 12" ------------------------------------------------------------------------------- 10.10.0.17:12 <=> 10.10.0.10:56536 160b 2.17kb 555b 10.10.0.17:12 <=> 10.10.0.10:36030 0b 1.68kb 431b 10.10.0.17:12 <=> 10.10.0.10:1692 160b 160b 194b ------------------------------------------------------------------------------- TX: cum: 0B peak: 0b rates: 0b 0b 0b RX: 15.0kB 11.0kb 160b 4.00kb 1.78kb TOTAL: 15.0kB 11.0kb 160b 4.00kb 1.78kb
Tip: use “t” to cycle through the four line display modes; the default 2-line display, with sent and received traffic on separate lines, and 3 1-line displays, with sent, received, or total traffic shown.
Related Posts
Finding Open Ports and Listening Services on Linux: Nmap, Netstat, lsof and ss
Measure and Troubleshoot Linux CPU Resource Usage
Measure and Troubleshoot Linux CPU Resource Usage
Measure and Troubleshoot Linux Disk I/O Resource Usage
Isn’t SSH port 22?
The default SSH port is TCP 22, however, I used to use a non-default port TCP 12 until I moved everything behing a VPN.
13:05:01 IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s
13:15:01 lo 22.17 22.17 6.04 6.04 0.00 0.00 0.00
13:15:01 eth0 144.48 149.90 0.00 0.00 0.00 0.00 0.00
13:25:01 lo 27.09 27.09 34.76 34.76 0.00 0.00 0.00
13:25:01 eth0 159.49 157.17 0.00 0.00 0.00 0.00 0.00
13:35:01 lo 21.91 21.91 6.13 6.13 0.00 0.00 0.00
13:35:01 eth0 141.16 146.25 0.00 0.00 0.00 0.00 0.00
13:45:01 lo 20.91 20.91 5.85 5.85 0.00 0.00 0.00
13:45:01 eth0 137.24 143.68 0.00 0.00 0.00 0.00 0.00
13:55:01 lo 26.50 26.50 39.18 39.18 0.00 0.00 0.00
13:55:01 eth0 148.30 151.32 0.00 0.00 0.00 0.00 0.00
Average: lo 23.71 23.71 18.39 18.39 0.00 0.00 0.00
Average: eth0 146.13 149.66 0.00 0.00 0.00 0.00 0.00