Increasing Performance of the Billing System

Very often it is necessary to receive and process large amounts of NetFlow packets from routers. For these packets not to be lost in case of overflow, it is necessary to enlarge the receive buffer by adjusting the configuration file ‘utm5.cfg

nfbuffer_bufsize=10485760

This directive sets the receive buffer size to 10 MB. At a start up of the billing system basic module, ‘debug.log’ should contain a line concerning successful enlargement of the receive buffer:

?Debug : Jan 29 19:39:30 NFBuffer: Setting SO_RCVBUF to <10485760> bytes

In some cases it is necessary to increase the maximum allowable buffer size in the operating system. It is possible to carry out this operation on Linux by using the following command:

sysctl -w net.core.rmem_max=10485760

on FreeBSD:

kern.ipc.maxsockbuf=10485760
net.inet.udp.recvspace=10485760
net.local.dgram.recvspace=10485760
net.inet.udp.maxdgram=100000

As a result of these settings the billing system will be able to receive large volumes of NetFlow packets. In particular, there will be no packet loss during peak stress.
For decreasing CPU usage and occupied hard disk space, it is necessary to use a new technique of traffic data storage. Detailed data are a full copy of NetFlow packets, received by the system. For switching the new technique on, it is necessary to add the following option in the ‘Settings’ area:

raw_storage_file=1

It is necessary to restart the system after addition of this parameter.
Detailed data will be stored in files with ‘.utm’ extension. This file format was developed by NetUP. ‘get_nf_direct’ utility can be used for further analysis of data, e.g.:

/netup/utm5/bin/get_nf_direct -b /netup/utm5/db/iptraffic_raw_1138443603.utm -u

The execution of this command results in output of the data saved in the file ‘/netup/utm5/db/iptraffic_raw_1138443603.utm
The basic module of the billing system saves the detailed data into a file during one hour. Then the file is closed and a new one is opened. An external program ‘/netup/utm5/bin/raw_fd_script’ is called, and a path to the closed file is transmitted to the program as a parameter. The program does all necessary actions: archiving, reserve copying, etc.
An example of the program ‘/netup/utm5/bin/raw_fd_script’ written on shel

#!/bin/sh
gzip $*

As a result of these settings the billing system will use recourses of CPU and disk subsystem optimally.
The billing system was tested under stress at NetUP test stand. Specification of the server used:

CPU: Intel Pentium 4 frequency 3.00 GHz
Memory: 2 GB
Hard disk: Serial ATA 250 GB
Netcard: Intel 100 Mbps
OS: Gentoo Linux

Another server was used to create a test stream of NetFlow packets. The packets were generated by the utm5_flowgen utility. CPU usage was monitored by the top utility. For better simulation NetFlow packets contain statistics concerning only registered test users. Correctness of receiving and processing of data was monitored by the traffic report and data from statistics.

Results of the test are represented on the diagram below:

According to observations of NetUP specialists, a totally loaded 100 Mbps channel in a real network generates statistics stream of about 100 NetFlow packets per second. Calculations show, that 3000 NetFlow packets per second correspond a 3 Gbps channel.