On Fri, Nov 20, 2009 at 4:51 PM, Mark Miller <mark...@gmail.com> wrote:
Okay - my fault - I'm not really talking in terms of Lucene. Though even
there I consider it possible. You'd just have to like, rewrite it :) And
it would likely be pretty slow.
Rewrite it how? When you index the very first document, the docFreq of all
terms is 1, out of numDocs = 1 docs in the corpus. Everybody's idf is the
No matter how you normalize this, it'll be wrong, once you've indexed a
documents. This isn't a matter of Lucene architecture, it's a matter of idf
a query-time exactly available value (you can approximate it partway through
indexing, but you don't know it at all when you start).