A novel term weighting scheme based on discrimination power obtained from past retrieval results

Cited 8 time in webofscience Cited 0 time in scopus
  • Hit : 418
  • Download : 0
Term weighting for document ranking and retrieval has been an important research topic in information retrieval for decades. We propose a novel term weighting method based on a hypothesis that a term's role in accumulated retrieval sessions in the past affects its general importance regardless. It utilizes availability of past retrieval results consisting of the queries that contain a particular term, retrieved documents, and their relevance judgments. A term's evidential weight, as we propose in this paper, depends on the degree to which the mean frequency values for the relevant and non-relevant document distributions in the past are different. More precisely, it takes into account the rankings and similarity values of the relevant and non-relevant documents. Our experimental result using standard test collections shows that the proposed term weighting scheme improves conventional TF*IDF and language model based schemes. It indicates that evidential term weights bring in a new aspect of term importance and complement the collection statistics based on TF*IDF. We also show how the proposed term weighting scheme based on the notion of evidential weights are related to the well-known weighting schemes based on language modeling and probabilistic models. (C) 2012 Elsevier Ltd. All rights reserved.
Publisher
ELSEVIER SCI LTD
Issue Date
2012-09
Language
English
Article Type
Article
Keywords

INFORMATION

Citation

INFORMATION PROCESSING & MANAGEMENT, v.48, no.5, pp.919 - 930

ISSN
0306-4573
DOI
10.1016/j.ipm.2012.03.004
URI
http://hdl.handle.net/10203/102324
Appears in Collection
CS-Journal Papers(저널논문)
Files in This Item
There are no files associated with this item.
This item is cited by other documents in WoS
⊙ Detail Information in WoSⓡ Click to see webofscience_button
⊙ Cited 8 items in WoS Click to see citing articles in records_button

qr_code

  • mendeley

    citeulike


rss_1.0 rss_2.0 atom_1.0