Detecting salient trends through a ranking method using trend line properties

Abstract

Automatic trend analysis from a collection of time-stamped documents, like patents, news papers, and blog pages, is a challenging research problem. Past research in this area has mainly focused on showing a trend line of a given concept over time by considering trend-associated term frequency information. Emerging trends were detected by checking a simple criterion such as frequency change or by recognizing a deviation from ordinary curves. We note that in order to show most salient trends detected among many possibilities, it is critical to devise a ranking function for trend lines. To this end, we define four properties of trend lines drawn from frequency information, to quantify various aspects of trends, and propose a method by which trend lines can be ranked. The properties are examined individually and in combination in a series of experiments for their validity using the ranking algorithm. The results show that a judicious combination of the four properties is a better indicator for salient trends than any single criterion used in the past for ranking or detecting emerging trends.