Antivirus Fight Club
Written by Karl L. Gechlik | AskTheAdmin.com on August 15, 2007 – 10:51 am -
Found this interesting article on ClamAV’s website.
Here’s the interesting tidbit:
The “winners” in last night’s contest were ClamAV, Kaspersky and Symantec. All three tools caught 100 percent of the viruses they encountered. F-Prot and Sophos caught 94 percent; McAfee caught 89 percent; and GlobalHauri, Fortinet, and SonicWall caught 61 percent.
I am not shocked to see Symantec where it is, because let’s face it…it is still best of breed.
But McAfee which is considered to be one of the big dogs to be so low is awful.
But a free, open source product in this test coming in @ 100% accuracy is VERY nice and applies some pressure.
So what are you using for antivirus??? Might want to think twice about it now.
EDIT: Rule Number One Of Anti Virus Fight Club - Never TALK about Anti-virus Fight Club!!!
Tags: General
Posted in General |
By Karl L. Gechlik on Aug 15, 2007 | Reply
Wow I’m Shocked. We might just need to rethink our AV strategy! Good looking out.
By NONE on Aug 15, 2007 | Reply
im not surprised i hate them nai peopel
By Anonymous on Aug 15, 2007 | Reply
The testing methodology has been heavily criticised by many members of the anti-virus community and respected security testing bodies such as Virus Bulletin, ESET (makers of NOD32), Independent researcher David Harley (PDF format), McAfee, Eddy Williems of EICAR, and Mark Harris of SophosLabs.
The ESET and David Harley papers in particular go into details as to why Antivirus Fight Club is probably not something worth paying much attention to.
By NinJaAdmiN on Aug 15, 2007 | Reply
Yes that last comment is true:
Whilst I’m sure the tests were well intentioned, they are not particularly scientific and as the author admits, Sophos results were significantly improved if the tester turned on the relevant options, suggesting the product documentation was not consulted.
Having retrieved the samples (the author having posted the malware samples on a public website!!!) it appears a few extra settings were required, in particular, one of the samples was a email stored as MIME but the ‘decode MIME’ option wasn’t turned on. Another sample was in fact a potentially unwanted application and again, the option to enable detection for PUAs wasn’t used.
While Sophos performed better than many of its competitors, the sample set was far too small, the methodology was confused and the author obviously isn’t well versed in handling malware. All of this goes to show that testing anti-virus products is a lot more complex than grabbing a few samples and scanning them.
I would recommend anyone wishing to compare products to look for recognised testing organisations and publications such as West Coast Labs and Virus Bulletin.
So take particular care in using this biased testing.
Do you just setup your AV and not configure the options?
By The Slothman on Aug 15, 2007 | Reply
ninja & anon: If you’re right, and some settings needed to be configured, then I would call that a fair test.
Simply put, most average people will simply install a product and assume it is set right. They are not uber-admins like us who tinker and dick around until we have things tight.
And no test is truly unbiased. It just isn’t. It’s like statistics, you pretty much always end up with the result you were looking for by excluding things that you don’t want to muck up the works.
By Karl L. Gechlik on Aug 15, 2007 | Reply
Sloth is right - it is not a measure of the product but how the normal user will install it.
Dude buys software
Dude installs software
Dude forgets about software until it is time to renew
Dude is your average end user.
By Anonymous on Aug 15, 2007 | Reply
why can i talk about antivirus fight club
By Joe admin on Aug 15, 2007 | Reply
thank you for the great story. i love reading your blog please keep up the amazing stories.