Free tool – PVE pcap statistics collector | Corelan Cybersecurity ResearchCorelan Cybersecurity Research

Free tool – PVE pcap statistics collector

Published May 25, 2008 |

Keywords : Wireshark Out Of Memory tcpdump statistics pcap

The story I’m about to tell may sound familiar to most of you. You’ve been asked to gather some bandwidth utilization statistics on one of your servers. You’ve installed Wireshark on the server to capture all traffic, or you’ve used linux’ tcpdump to capture all traffic. You’ve tried to open the capture file in Wireshark so you could look at the bandwidth utilization… but Wireshark doesn’t like your big file and dies with the following message :

Out Of Memory!

Sorry, but Wireshark has to terminate now!

(Well, in fact, Wireshark is not to blame here – it’s just your system that cannot handle that amount of data in memory)

I came across this scenario many times. And I decided to write a little tool that will just read the file and dump out the statistics, without putting a huge burden on your memory. After all, I don’t care about the packet details at this time… I just want to gather some basic statistics.

The tool is called PVE pcap statistics collector and can be downloaded from http://users.telenet.be/internet.activities/freetools/pvepcapstats/index.htm

As usual – the tool is free.

Note : if you need more info from the wireshark capture files, please have a look at this document and look at the tshark tool

This is how it works :

First of all, install the application. Make sure you have winpcap installed as well.

The application is a command line utility, so you’ll need to open a command prompt and go to the path where the tool is installed. Usually, this will be C:\Program Files\Peter Van Eeckhoutte\PVEpcapStats\

C:\Program Files\Peter Van Eeckhoutte\PVEpcapStats>dir
 Volume in drive C has no label.
 Volume Serial Number is 585F-8B81

 Directory of C:\Program Files\Peter Van Eeckhoutte\PVEpcapStats

25/05/2008  15:47              .
25/05/2008  15:47              ..
25/05/2008  15:29            53.248 PVEPcapStats.exe
25/05/2008  15:29           159.744 PVESharpPcap.dll
               2 File(s)        212.992 bytes
               2 Dir(s)  26.953.371.648 bytes free

You can launch the tool by running "pvepcapstats.exe"

If you don’t provide any command line arguments, you’ll get the banner and some syntax information :

C:\Program Files\Peter Van Eeckhoutte\PVEpcapStats>PVEPcapStats.exe

---------------------------------------------------------
PVE Wireshark/tcpdump statistics collector
Written by Peter Van Eeckhoutte
https://petersblog.dyndns.org:8899
Version 1.0.0.6
---------------------------------------------------------

  Usage : PVEPcapStats.exe -f 
     Optional arguments :
       -s  -e 
          Put both the starttime and endtime values between double quotes
          This setting will force the application to only look at packets
          that were captured between  and 
       -m 
          If you specify the -m parameter, followed by a number
          then a txt textfile (tab separated) will be created, containing some
          statistics, grouped by the specified number of minutes
       -p 
          The -p parameter allows you to filter on port number(s)
          The statistics will only be based on these TCP/UDP port numbers
          The port numbers (if multiple) must be comma separated.
          Don't put spaces between the ports !

You may notice that the application appears to delay right after "Version…". This is because, at that time, it will attempt to see if there is a newer version available.

What do the arguments mean ?

First of all, you need to feed the name of the wireshark/tcpdump file to the application. Use the parameter -f to specify the filename. If the filename contains spaces, make sure to put the filename between double quotes.

If you run the tool with just the -f parameter, you’ll get the overall statistics for the entire pcap file, including the date & time of the first and last packet.

C:\Program Files\Peter Van Eeckhoutte\PVEpcapStats>PVEPcapStats.exe -f "tcpdump capture.cap"
---------------------------------------------------------
PVE Wireshark/tcpdump statistics collector
Written by Peter Van Eeckhoutte
https://petersblog.dyndns.org:8899
Version 1.0.0.6
---------------------------------------------------------
  Parameters detected :
  - Pcap file :
    tcpdump capture.cap

[+] Please wait while processing Wireshark/tcpdump file...

[+] Statistics overview :

  tcpdump file size                         : 1073746328 Bytes
                                              (1024 MBytes)
  First packet timestamp                    : 15/05/2008 14:53:30
  Last packet timestamp                     : 15/05/2008 15:58:49
  Nr of secs between first and last packet  : 3919
  Nr of packets read                        : 988459/988459
  Total size of all analyzed packets        : 1057930960 Bytes
  Average packet size                       : 1070,28 Bytes
  Average Bytes/sec                         : 269949,212
          Megabits/sec                      : 2,060

If you are only interested in a specific part of the libpcap file, you can specify a starttime and endtime (Date + time in both cases. When specifying a start and end date/timestamp, make sure to verify the format of the Date & Time string, so it matches the format of the output on your screen). All other data in the libpcap capture file will be ignored. In order to set this up, you’ll need to use the -s (start time) and -e (end time) parameters. Put both date×tamps between double quotes.

Make sure not to specify timestamps that are outside of the captured file contents. Otherwise, the calculated average bandwidth will be wrong.

You can use the the -m parameter to create a tab separated txt file, that contains bundled statistics, grouped by the number of minutes you’ve specified. So if you logfile contains 1 hour of data, and if you specify -m 10, then you’ll get a txt file with 6 entries. Every entry will have the average bandwidth, packetsize, etc, for these 10 minutes.

By default, if the file bandwidthstats.txt does not exist, a new file will be created. If the file does exist, info will be appended. Don’t forget to move/rename older txt files if you want to perform stats on a new set of data files.

Last but not least, you can tell the application to only look at traffic that contains certain ports. You can only filter on tcp/udp ports. You can specify multiple ports by comma-separating the port numbers. Make sure not to put any spaces between the various port numbers.

An example :

Suppose your capture file is called "tcpdump capture.cap" and is located in the same folder as the application. In order to get the start and end time of the capture file, you could run the tool with only the -f parameter (see above). Based on this information, you can further refine the statistics collection task.

If you are only interested in the packets between 14:55 and 15:55, and you want to get a txt file with the statistics for every 5 minutes in that time window, and if you only want to see traffic that uses port 1352 and port 25, then this would be the command line you need to use :

C:\Program Files\Peter Van Eeckhoutte\PVEpcapStats>PVEPcapStats.exe -f "tcpdump capture.cap"

  -s "15/05/2008 14:55:00" -e "15/05/2008 15:55:00" -m 5 -p 1352,25

---------------------------------------------------------
PVE Wireshark/tcpdump statistics collector
Written by Peter Van Eeckhoutte
https://petersblog.dyndns.org:8899
Version 1.0.0.6
---------------------------------------------------------
  Parameters detected :
  - Pcap file :
    tcpdump capture.cap
  - Only process packets that have been captured between :
      15/05/2008 14:55:00
      and
      15/05/2008 15:55:00
  - Use output file with bandwidth statistics, bundled into
    windows of 5 minutes
    ** If file 'bandwidthstats.txt' already exists, data will be appended ! **
  - TCP/UDP port(s) to look at : 1352,25

[+] Please wait while processing Wireshark/tcpdump file...

[+] Statistics overview :

  tcpdump file size                         : 1073746328 Bytes
                                              (1024 MBytes)
  First packet timestamp                    : 15/05/2008 14:53:30
  Last packet timestamp                     : 15/05/2008 15:58:49
  Analysis start timestamp                  : 15/05/2008 14:55:00
  Analysis end timestamp                    : 15/05/2008 15:55:00
  Nr of secs between first and last packet  : 3600
  Nr of packets read                        : 909873/988459
  Total size of all analyzed packets        : 976218629 Bytes
  Average packet size                       : 1072,92 Bytes
  Average Bytes/sec                         : 271171,841
          Megabits/sec                      : 2,069

The bandwidthstats.txt looks like this :

If you want to analyze the entire file, and create a bandwidthstats file that contains averages for every minute, use this command :

C:\Program Files\Peter Van Eeckhoutte\PVEpcapStats>PVEPcapStats.exe -f "tcpdump capture.cap" -m 1
---------------------------------------------------------
PVE Wireshark/tcpdump statistics collector
Written by Peter Van Eeckhoutte
https://petersblog.dyndns.org:8899
Version 1.0.0.6
---------------------------------------------------------
  Parameters detected :
  - Pcap file :
    tcpdump capture.cap
  - Use output file with bandwidth statistics, bundled into
    windows of 1 minutes
    ** If file 'bandwidthstats.txt' already exists, data will be appended ! **

[+] Please wait while processing Wireshark/tcpdump file...

[+] Statistics overview :

  tcpdump file size                         : 1073746328 Bytes
                                              (1024 MBytes)
  First packet timestamp                    : 15/05/2008 14:53:30
  Last packet timestamp                     : 15/05/2008 15:58:49
  Nr of secs between first and last packet  : 3919
  Nr of packets read                        : 988459/988459
  Total size of all analyzed packets        : 1057930960 Bytes
  Average packet size                       : 1070,28 Bytes
  Average Bytes/sec                         : 269949,212
          Megabits/sec                      : 2,060

Open the txt file in a tool such as MS Excel and you’ll be able to create some nice graphs about your bandwidth utilization.

Enjoy – and let me know if something doesn’t work as expected…

Note : in order to interprete the results correctly, make sure to verify your decimal/thousands separator settings on your computer. On my computer, the regional settings are set to , = decimal separator.

Posted in 001_Security, My Free Tools, Networking | Tagged bandwidth, calculate-bandwidth-of-pcap, claculate-pcap-file-size, create-statistics-from-pcap, free tool, network utilization, out of memory, pcap, pcap-file-packet-size, pcap-file-statistics, pcap-generate-stat, pcap-statistics, psexec-v1-83-suite, statistic-file-pcap, statistics, statistics-pcap, stats-from-pcap, stats-pcap, tcpdump, wireshark

Corelan Cybersecurity Research

:: Knowledge is not an object, it's a flow ::