Design and implementation of a logging interface for an open content delivery interconnection system

Abstract

It is expected that in three years, the amount of Internet traffic will quadruple and ninety percent of this trafic is video content. In order to maintain an acceptable quality of service to end-users, Network Service Providers (NSP) deploy their own content distribution networks (CDN). However, NSP`s CDN is limited by its own network footprint. One way to expand the NSP`s CDN footprint is to interconnect CDNs (CDNi) from different service providers such that multiple autonomous CDNs can operate as a combined infrastructure for the delivery of content from content service providers to the end-users.

In this thesis, we developed a suitable logging infrastructure for a CDNi system. The logging interface is important to provide debugging and accounting functionalities to the CDNi. The primary objective of the logging infrastrucutre is to exchange information, represented as log files, between participating CDNs. In order to develop a unified logging infrastructure for CDNi applications, we divided the logging problem into three distinct parts: log generation, log transport, and log analysis. In addition, extracting useful information from text log files was explored. Text log files are composed of a sequence of log entries, where each log entry is represented as a stream of characters. By integrating data stream monitoring techniques and dynamic metadata generation, we were able to improve extraction of desired information from text files with query time comparable to popular relational database management system.