Author Topic: Malware monitoring platform  (Read 8140 times)

0 Members and 1 Guest are viewing this topic.

September 07, 2009, 09:22:36 am
Read 8140 times

Malware-Web-Threats

  • Special Members
  • Hero Member

  • Offline
  • *

  • 354
    • MalwareURL
Did you never think of creating a platform to monitor websites listed on MDL?

For example all websites that redirects to other malicious files/urls. Everytime I check a website from the MDL list I found a new one.

It will be useful for you to monitor them in an automated way. Now that you have clean-mx with a great platform it's time to create yours.  ;D

jsunpack is available to download if you want to analyze javascript, pdf files etc.. The script is in python (I'm not familiar with this language so I can't help with this)

Wepawet can be used to send urls, pdf, flash, executable and we can retreive VirusTotal (% and virus names), Anubis (traffic capture and payload behaviors), ThreatExpert, Sunbelt, Prevx and tons of useful information.

I can also provide some scripts to check for redirections (headers redirects, javascript etc.)

September 07, 2009, 09:51:53 am
Reply #1

SysAdMini

  • Administrator
  • Hero Member

  • Offline
  • *****

  • 3335
I'm  very sceptical about automated malware monitoring.

In my opionion there is no reliable way to determine automatically if an url is malicious or not.
There are too many dependencies. Think about the Fragus kit where you first have to visit the show.php before you get access to the payload.
Other sites require special useragents or are only available to special geographic regions.
Some sites return http code 404, but deliver exploits anyway.

There are too many things to consider. Therefore you can't just pull all urls from db and try to download the content.
This is the reason why I don't do it for MDL. It has the disadvantage that a lot of MDL urls are already inactive or cleaned, but still listed as active.

I appreciate your idea in general, but I'm sceptical. And there is another drawback. You might run into legal issues if you flag sites as malicious mistakenly.
This is the reason why I check all urls manually before I add it to list and why I don't simply import submissions.
And even you do it this way, you can run into problems when you call a software "Rogue" if it is Adware. I have experienced too many of those cases.
Ruining the bad guy's day

September 07, 2009, 10:04:48 am
Reply #2

Malware-Web-Threats

  • Special Members
  • Hero Member

  • Offline
  • *

  • 354
    • MalwareURL
Quote
In my opionion there is no reliable way to determine automatically if an url is malicious or not.
You have not totally wrong :-o

Quote
Other sites require special useragents or are only available to special geographic regions.
I use a US, UK IPs, some double check with Google, Yahoo and Bing as referer and have no problem with this.

Quote
Some sites return http code 404, but deliver exploits anyway
Most of them is because you must be redirect from another site to sucessfully load the exploit (referer needed)

Quote
You might run into legal issues
Not if you protect yourself with Terms and Conditions. Read the disclaimer for example: www.malwarepatrol.com

-------

And what about Threat Expert? (advanced automated threat analysis system)

September 07, 2009, 10:12:59 am
Reply #3

Orac

  • Special Members
  • Hero Member

  • Offline
  • *

  • 723
    • malwareremoval.com
I agree with SysAdMini. There are way to many variables involved, until true AI becomes a reality i dont believe it will be possible to achive an effective, reliable automated method that can come close to matching, let alone beating the old mark one eyeball for the job.

Ive been involved with two attempts to automate the process, both ended up in dismal failure, dispite one of them being lead by someone who had done a similar think in realtion to phising sites which worked vey effectively.

I look forward one day to being proved wrong, until then its the old fashioned way iam afraid.
Malware analysised using clarified analyzer to record and document how malware behaves in a networking environment

September 07, 2009, 10:23:06 am
Reply #4

Malware-Web-Threats

  • Special Members
  • Hero Member

  • Offline
  • *

  • 354
    • MalwareURL
Quote
There are way to many variables involved

Yes may be but is this for the 10% of sites that you can't load in an automated way that you abandon the idea?

You must rethink - just take a look:

malwareurl.com

Most of them are added in an automated way.

September 07, 2009, 10:33:42 am
Reply #5

cleanmx

  • Special Members
  • Hero Member

  • Offline
  • *

  • 3405
    • Spam-Filter Anti-Spam Virenschutz - CLEAN MX Managed Anti-Spam Service ist die Lösung für Ihr Spam-Problem
hi @all

I agree not all urls may be added automagically...
but as malwareurl do, so dow we at cleanmx.

we analyse threatexpert an anubis, we analyse links inside retrieved url's and we do some other vodoo....

I suppose less than 5% will be left over to be reviewed by human eye.

and legal.... i have no problems, I only collect these evil, and complain about them, i will never block someone....

I pers. think we should concentrate us on these 5% and we shall integrate our databases .... just in the way I do it for now on clean-mx !

malwaredomainlist=sub4 in my database
malwareurl=sub6

I wrote a request to http://www.malwarepatrol.net/ for dataexchange.
and I will reactive google.... theay stoped after a couple of transmissions...

-- gerhard

September 07, 2009, 10:47:36 am
Reply #6

SysAdMini

  • Administrator
  • Hero Member

  • Offline
  • *****

  • 3335
and legal.... i have no problems, I only collect these evil, and complain about them, i will never block someone....

It seems that you haven't got letters from lawyers.
Ruining the bad guy's day

September 07, 2009, 10:49:00 am
Reply #7

cleanmx

  • Special Members
  • Hero Member

  • Offline
  • *

  • 3405
    • Spam-Filter Anti-Spam Virenschutz - CLEAN MX Managed Anti-Spam Service ist die Lösung für Ihr Spam-Problem
Quote
It seems that you haven't got letters from lawyers.

no never since 2004 !

September 07, 2009, 11:29:20 am
Reply #8

RS-232

  • Special Access
  • Sr. Member

  • Offline
  • *

  • 165
Quote
And there is another drawback. You might run into legal issues if you flag sites as malicious mistakenly.
....
And even you do it this way, you can run into problems when you call a software "Rogue" if it is Adware.
I have experienced too many of those cases.
...and as far as I can remember,JohnC also had ran into numerous such cases of complaints and legal threats...

Quote
Not if you protect yourself with Terms and Conditions. Read the disclaimer for example: www.malwarepatrol.com
From their FAQ:
Quote
# Can I get an unsanitized list of URLs?
We do not make unsanitized URLs public. If you have a real need for it, please contact us.
We exchange such lists with CSIRTs and known security groups.
...if the goal is to provide blocklists without full url links but just with the domain names,
then there are quite a few projects already doing so succesfully...why re-invent the wheel here?...

Still though,i agree that some kind of semi-automation/synchronization or even integration would be nice,
if it was to take place between the databases,ie.MDL,hpHosts,CleanMX etc etc...
But who has the knowledge/patience to code that and how,that certainly needs quite a lot of conversation...
Only for the "fun" of it...rs-232 aka sowhat-x aka younameit ;-)
http://www.youtube.com/watch?v=fADjY97_KTw

September 07, 2009, 11:34:40 am
Reply #9

SysAdMini

  • Administrator
  • Hero Member

  • Offline
  • *****

  • 3335
Quote
And there is another drawback. You might run into legal issues if you flag sites as malicious mistakenly.
....
And even you do it this way, you can run into problems when you call a software "Rogue" if it is Adware.
I have experienced too many of those cases.
...and as far as I can remember,JohnC also had ran into numerous such cases of complaints and legal threats...


Oh yes, I can confirm that.
Ruining the bad guy's day

September 07, 2009, 11:39:58 am
Reply #10

Malware-Web-Threats

  • Special Members
  • Hero Member

  • Offline
  • *

  • 354
    • MalwareURL
I already have all the stuff to do this.

Do you have tools if I send you a list of 1.000 exploits domains?

This can be done in an automated way using wepawet for example then based on a score we can add them on a private list for analysis (if exploits found, if urls leads to exe or pdf, if exe is detected with more than 4/40 on VT etc..)

September 07, 2009, 11:51:56 am
Reply #11

cleanmx

  • Special Members
  • Hero Member

  • Offline
  • *

  • 3405
    • Spam-Filter Anti-Spam Virenschutz - CLEAN MX Managed Anti-Spam Service ist die Lösung für Ihr Spam-Problem
shure ... i have tools... knife... hammer...

-- gerhard

update this was inteded to be a little joke  ;)

September 07, 2009, 12:53:54 pm
Reply #12

sparsha

  • Special Members
  • Hero Member

  • Offline
  • *

  • 305
Anthony

Your idea will be excellent for the rotators!

September 07, 2009, 03:26:16 pm
Reply #13

Malware-Web-Threats

  • Special Members
  • Hero Member

  • Offline
  • *

  • 354
    • MalwareURL
what do you mean by "rotators"?

September 07, 2009, 07:12:01 pm
Reply #14

cleanmx

  • Special Members
  • Hero Member

  • Offline
  • *

  • 3405
    • Spam-Filter Anti-Spam Virenschutz - CLEAN MX Managed Anti-Spam Service ist die Lösung für Ihr Spam-Problem
hi Anthony,

I suppose he meant those  using (fast)-flux technologie (rotating ip's nameserver's etc...)

-- gerhard