tf-idf weighting

2019-05-29 00:10:25 浏览数 (1)

版权声明:本文为博主原创文章,转载请注明出处。 https://cloud.tencent.com/developer/article/1437731

水平有限,如有错误请指正!

tf-idf weighting

tf(term frequency)

a document or zone that mentions a query term more often has more to do with that query and therefore should receive a higher score

query term: a set of words

NN:document中term的总数

tt:document中term出现的次数

tft=tN

tf_t=frac{t}{N}

idf(inverse document frequency)

NN: document的数量

dftdf_t:有多少documents 包含tt

idft=logNdft

idf_t=logfrac{N}{df_t}

从公式可以看出:

  • dftdf_t 越小,idftidf_t越大,表明tt对文档的区分性更大
  • 反之

tf-idf weighting

tf-idft,d=tft,d∗idft

text{tf-idf}_{t,d}=tf_{t,d}* idf_t

参考:

tf

idf

tf-idf

0 人点赞