版权声明:本文为博主原创文章,转载请注明出处。 https://cloud.tencent.com/developer/article/1437731
水平有限,如有错误请指正!
tf-idf weighting
tf(term frequency)
a document or zone that mentions a query term more often has more to do with that query and therefore should receive a higher score
query term
: a set of words
NN:document中term的总数
tt:document中term出现的次数
tft=tN
tf_t=frac{t}{N}
idf(inverse document frequency)
NN: document的数量
dftdf_t:有多少documents 包含tt
idft=logNdft
idf_t=logfrac{N}{df_t}
从公式可以看出:
- dftdf_t 越小,idftidf_t越大,表明tt对文档的区分性更大
- 反之
tf-idf weighting
tf-idft,d=tft,d∗idft
text{tf-idf}_{t,d}=tf_{t,d}* idf_t
参考:
tf
idf
tf-idf