Jeremy Bowers has outlined an attack against Bayesian spam filtering (via k5).
Jeremy’s premise is that, for a given language, everyone’s non-spam corpora will be very similar. Spammers can exploit this liguinstic similarity by building their own corpora, identifying spam words and replacing them with words that have a higher non-spam probability.
Jeremy demonstrates this using a variant of the Nigerian scam, one which my own installation of POPFile had never encountered. I mailed his original and modified versions to myself, the original was classified as spam while the modified version was not.
The threat from this attack comes from the process of correcting these missed spams. Adding them to a spam corpus will cause it to more strongly resemble a non-spam corpus, which could result in higher rates for false negatives and false positives.
But I’m not ready to concede that this will be the end of the world. Jeremy admits that this technique may not work for sex spam:
It is indeed impossible to pitch a sex scam at someone who doesn’t really want it, unless you have some legitimate need to talk about sex, in which case you will have trouble with accurate filtering.
Nigerian scam variants differ from many other types of spam in that they already have a strong resemblance to normal English. My installation of POPFile did not catch the first Nigerian scam message it encountered, but it has caught most of them since. POPFile continues to work fine for me with few errors, and as I already said, it caught the original version of Jeremy’s Nigerian letter despire having never received that particular variant before.
Most of my spam isn’t trying to sweet talk a wire transfer out of me, it’s trying to sell me something (legit or not). Sex. Drugs. Toys. Contests. Wealth. I’m not convinced that these types of spam are easily modified to resemble normal English without compromising the message, and I’m not sold on the long-term harm to my corpora.
Sex spam can’t escape Bayes because there are no synonymous words for sex acts left to use that occur with great frequency in normal mail. Why would the long-term result be any different for herbal viagra, furby, 42″ plasma television give-aways, and get rich on eBay schemes?
True ture,
Bayesian does have it’s limits, but used in conjunction with Heuristics, distributed checksum clearinghouse, and RBLs the success rate is much higher. I work for an ASP, that uses these four methods and we’re not stopping 99% with zero false positives.