Flow Based Dynamic Load Balancing for Passive Network Monitoring

Abstract

Cluster based packet capturing is a way of overcoming the speed of a slow disk to tap a high-speed network. Most cluster-based architectures, however, do not consider load balancing as an important issue. In order to perform mon itoring at full line speed without losing packets, we accept that the balance among back-end servers must be main tained. Conventional methods rely on fixed or random rout ing where a ”source-destination” address pair is used as a unit for load balancing. Instead of using such a coarse grain unit, we use a flow, a ”source-destination-port” pair. In ad dition, we adaptively balance loads of back-end servers us ing a flow load estimation technique. The proposed meth ods have been validated by performing a trace-based sim ulation. Compared to the existing routing approaches, our method balances load of back-end servers. From a simula tion, we find that for protocols having a transient load with a small variance in flow length, we may use simple meth ods of load balancing such as round-robin, but for those having a persistent load with a large variance, we need to use a flow estimation technique.