Comparison of enterprise anti-virus solutions

[Secured Sector]

Just thought this might be of interest for you: http://www.securedsector.com/forumdisplay.php?fid=6

You will find there a comparison of enterprise anti-virus solutions and their malware detection capabilities.

[sowhat-x]

...welcome on board! 

Sure it's of interest...actually,there was already some discussion,
regarding reliability and/or methodology of AV comparison tests,
in the following thread over at Offensive Computing...
a few people lurking around here commented there as well... 
http://www.offensivecomputing.net/?q=node/537

[Secured Sector]

...welcome on board! 

Thank you.

Sure it's of interest...actually,there was already some discussion,
regarding reliability and/or methodology of AV comparison tests,
in the following thread over at Offensive Computing...
a few people lurking around here commented there as well... 
http://www.offensivecomputing.net/?q=node/537

Thanks for pointing me to this. Well, as I mentioned in my post here I am an administrator and not a virus researcher or something else and in no way I want to compete with av-comparative.org or whatever.

By the way: Someone mentioned that the malware archive contains only old malware stuff and it would not make sense to use it for tests. Sure, there are some old and DOS based viruses, but I have excluded them from my test set. However, other malware test sets are highly appreciated for my next tests and if someone has more malware except the from malwaredomainlist.com repository I am willing to include these in my comparative.
 

[sowhat-x]

...he-he,someone has to be quite mad to follow this thread,
since it's actually splitted in...three different forums! 

Ok,quite a lot of things to say here...so allow me to quote the following words from you:

I am the admin in charge for McAfee VirusScan Enterprise for our company-wide network,
and I am very curious about the detection rates of the virus protection solution,
I am responsible for...

Thus,if I understand well,to say it in...plain words,
you're worried if McAfee misses a sample that finds it's way into the network,and then...
well,ok,we all know how it is to deal around with the boss when something gets wrong... 
Now,I'm most probably gonna disappoint you for a variety of reasons...

At first,let me say I'm also personally a McAfee Enterprise v8.0i user,
haven't bothered updating to v8.5... .dat definitions are the same though.
And honestly,neither I will "update" it anywhere in the near future...
I'm using it not because it's the "greatest" out there,
but because I feel confortable with it after all these years...
And obviously,because either we like this fact or not,you need to run some AV product after all...

Since it's quite obvious that you enjoy statistics,lol...here's a few:
Everything packed with Upack 0.39 final -> New Malware.aj
Everything packed with the most recent Upack 0.3999 -> Not detected at all
Everything packed with 7 or 8 older versions of Upack -> Not detected at all

Explanation of the above:
Some guy over at McAfee got bored of writing a proper unpacking engine for this popular packer,
and preferred to just flag as malware(!) the version that was currently at his hands,ie.v0.39 final.
Thanks God,they didn't have access to more versions as well! 

Want more statistics?...

BeroExePacker -> SDBot.worm.gen
Yeah,sure!No unpacking here also...this is actually a packer for the "demoscene".

RLPack 1.18/1.19 Basic -> Win32.New Virus
This version is also...open source!Bravo...
Not to mention that the full version of it is a commercial product...

Even more statistics?...
RCryptor 9 different versions -> only 2 detected...
About 250 malware packers in my personal repository (multiple versions) -> 99 samples detected...
Meaning...about 40% success.Is that so...why that much? 
Because half of them are packed with Upack 0.39 final mentioned above,he-he...
thereby detected as NewMalware.aj,lol... 
Unpack them...and nothing's detected at all.
So,you can safely trim down this result down to 20-25 % or so...
Wanna be more strict than the above results?
Ok,so let's shutdown McAfee temporarily,
and start packing trojans with the "detected" ones,under VMWare/DeepFreeze or so...
Guess what...the packer itself is detected...but the packed trojans are not,
wait,there's a name for this...advanced technology of malware detection,and it's called:
good old plain MD5-based hashing/fingerprinting...

Ok,let's say you're not interested in packers at all,
and that we should leave them out,even if 85-90 % of malware is packed nowadays...
Take a visit here,in the releases page:
http://hellknights.void.ru/
These guys have made available in public,a lot of them with sources as well,
more than a few rootkits,viruses,backdoors etc.
They're also considered to be one of the greatest "elite" russian vx team nowadays,
well-known and respected in a lot of reverse engineering and trojan-exchange forums.
I first came across their page somewhere in the late May.
Until late September,ALL of their releases were still undetected by McAfee.
Even worse,note that in the four months period in between,
I've submitted them via WebImmune at least 4-5 times...NADA.
I don't know if they've finally added detection for them:
after all these months and multiple submissions,I simply don't give a damn anymore...

I spoke about McAfee,because that's the one I'm experienced with...
and certainly NOT because I want to dis-advertise them.
For example,they generally seem to have an EXCELLENT detection rate,
when it comes to M$ 0-day exploit malware samples...

The point of all the above is...that exactly the SAME drawbacks,
stand true for EVERY AV product out there nowadays.
As already said,someone simply chooses to use the one that he/she likes to think,
that it will fit his NEEDS better,not the one that IS actually "better":
there never was,and will never be a reliable metric on which one is "better".
And as time passes,and malware becomes even more complex/sophisticated,
my guess is that old-style "comparison metrics" will become completely obsolete...

Hope the above statements were not too much disappointing...
hey,not everything is lost in this world!
But I also felt the need of commenting the following statement that you made,
for a wide variety of reasons...

I am an administrator and not a virus researcher or something else...

...well,this surely made me laugh...not in a "hostile" way,quite the contrary...
but because I'm also not an expert "security researcher"...
it's exactly because I had came in a position somehow similar with yours...
when the whole thing started for me...almost about a year ago,
I had to also work as a "second hand" helper aside our network admin here...
and I quickly realized that no matter how much you fiddle around with DMZ's ,routers etc,
if just one employee launches by accident a damned executable via clicking in his browser...
Soon I discovered the aformentioned facts about AV products,and well,here we are now,
learning more stuff,messing around with packers,Olly and backdoors... 

[Secured Sector]

Thus,if I understand well,to say it in...plain words,
you're worried if McAfee misses a sample that finds it's way into the network,and then...
well,ok,we all know how it is to deal around with the boss when something gets wrong... 

No, I donīt worry about the fact the even the most sophisticated anti-virus programs are unable to detect every single piece of malware because this is neary impossible. I just wanted to see that at least one security layer (anti-virus) works as expected...

Since it's quite obvious that you enjoy statistics,lol...here's a few:
Everything packed with Upack 0.39 final -> New Malware.aj
Everything packed with the most recent Upack 0.3999 -> Not detected at all
Everything packed with 7 or 8 older versions of Upack -> Not detected at all

Explanation of the above:
Some guy over at McAfee got bored of writing a proper unpacking engine for this popular packer,
and preferred to just flag as malware(!) the version that was currently at his hands,ie.v0.39 final.
Thanks God,they didn't have access to more versions as well! 

Yep, I am aware of modifying executables in such a manner that even with activated full heuristic capabilities they arenīt recognized as malware by virus-scanners.

Want more statistics?...

BeroExePacker -> SDBot.worm.gen
Yeah,sure!No unpacking here also...this is actually a packer for the "demoscene".

RLPack 1.18/1.19 Basic -> Win32.New Virus
This version is also...open source!Bravo...
Not to mention that the full version of it is a commercial product...

Even more statistics?...
RCryptor 9 different versions -> only 2 detected...
About 250 malware packers in my personal repository (multiple versions) -> 99 samples detected...
Meaning...about 40% success.Is that so...why that much? 
Because half of them are packed with Upack 0.39 final mentioned above,he-he...
thereby detected as NewMalware.aj,lol... 
Unpack them...and nothing's detected at all.
So,you can safely trim down this result down to 20-25 % or so...
Wanna be more strict than the above results?

This is quiet interesting. Could you provide me with a couple of malware packers? Maybe I am going to repeat my test with other test sets and modified binaries. I already performed some on-demand scan tests with a couple of files packed with nPack, BeroExe or upack just for fun and Avira AntiVir Classic didnīt recognize a modified worm while McAfee did.

However, I am sure that even VirusScan Enterprise with installed Antispyware Enterprise module will not detect every packed malware file.

Take a visit here,in the releases page:
http://hellknights.void.ru/
These guys have made available in public,a lot of them with sources as well,
more than a few rootkits,viruses,backdoors etc.
They're also considered to be one of the greatest "elite" russian vx team nowadays,
well-known and respected in a lot of reverse engineering and trojan-exchange forums.
I first came across their page somewhere in the late May.
Until late September,ALL of their releases were still undetected by McAfee.
Even worse,note that in the four months period in between,
I've submitted them via WebImmune at least 4-5 times...NADA.
I don't know if they've finally added detection for them:
after all these months and multiple submissions,I simply don't give a damn anymore...

Too bad that the forementioned site seems to be offline at the time being. I am really interested in testing some samples from their site - do you know if they own another web site?

I spoke about McAfee,because that's the one I'm experienced with...
and certainly NOT because I want to dis-advertise them.

I see.

For example,they generally seem to have an EXCELLENT detection rate,
when it comes to M$ 0-day exploit malware samples...

The point of all the above is...that exactly the SAME drawbacks,
stand true for EVERY AV product out there nowadays.

I donīt agree completely. If product A detects more malware stuff than product B and/or product C, which one would you are going to buy if you are a decision maker in your company? By the way: I am aware that there are many other factors when it comes to enterprise anti-virus solutions but I am just talking about detection rates.

As already said,someone simply chooses to use the one that he/she likes to think,
that it will fit his NEEDS better,not the one that IS actually "better":
there never was,and will never be a reliable metric on which one is "better".
And as time passes,and malware becomes even more complex/sophisticated,
my guess is that old-style "comparison metrics" will become completely obsolete...

Well, as I mentioned above anti-virus programs are just one security layer. I never ever would rely only on this security layer since I know that only many and independent security layers can offer some kind of protection for large enterprise scenarios.

Hope the above statements were not too much disappointing...

Absolutely not. In contrast, I want to learn and so I really appreciate other people meanings.