Discovery of highly interrelated communication entities in the communication log

Abstract

Recently many countries including the U.S. and the EU are legally forcing their communication service providers to retain electronic communication records, often called \emph{communication log}, for a certain amount of time. These retained communication logs are being used to prevent, investigate, detect, or prosecute serious crimes by the law enforcement agencies (LEAs) such as police, FBI, etc. In general, the communication logs rarely include whole communication content owing to privacy or technical issues; i.e., only minimum information such as senders, receivers, dates and times, locations, etc. is stored in the logs. In particular, one-way communication logs often include a huge amount of spam entities or spammers, which send unsolicited or undesired messages to numerous recipients via electronic messaging systems. This is because spammers can indiscriminately send their spam messages to any recipients by using one-way communication services such as e-mail, SMS, etc. if they only know the address of the recipients. In this dissertation, we propose \emph{score-based} and \emph{sequence-based} methods for finding highly interrelated communication entities from the one-way communication logs, even though the logs include many spam entities. A Spam-Robust Proximity Scorer, the score-based method, discovers highly interrelated communication entities from the one-way communication log by measuring the proximity scores of normal communication entities with respect to the \emph{surveillance target communication entities} (or just shortly surveillance targets(\) such as criminals, suspects, etc. In other words, for the given surveillance targets, the communication entities that get high proximity scores by the method are likely to be highly interrelated with the surveillance targets. To measure the proximity scores, we derived a new formula considering several metrics such as the number of adjacent communication entities, the number of incident communications, an...