Ted Leung pointed at Latent Semantic Indexing today, which got me reading some papers. The patent situation is unfortunate, because it is a pretty nice technique. Then, thanks to Technorati, I found this which led to Building a Vector Space Search Engine in Perl.
Now this isn’t quite latent semantic indexing, but it uses some of the same techniques. I’m not sure what the patent situation is - this seems fairly trivial, but who knows? Either way, this technique is really, really good for those times where you want to categorise text into a number of potential categories, mainly because it isn’t too resource intensive. Compared to Bayesian classification, it appears that the algorithm should be much, much quicker than even the best Bayesian implementation.
I figured that anything they can do in Perl, I can do in Java, so I’d like to present my very ugly Java version. This isn’t nice code, and it doesn’t do Stemming (note to self - look at using Lucene’s stemming code), it uses doubles instead of BigDecimals (and/or BitSets), but it appears to work. I haven’t done vector math for a long time, so I might have screwed that up somewhere, too.
However, it’s something I’m going to look at more in the future. I’ll probably build a classifier for Classifier4J based on it, and compare it to my Bayesian classifier.