Malware Domain List

Malware Related => Malware Analysis => Topic started by: Malware-Web-Threats on September 07, 2009, 09:22:36 am

Title: Malware monitoring platform
Post by: Malware-Web-Threats on September 07, 2009, 09:22:36 am
Did you never think of creating a platform to monitor websites listed on MDL?

For example all websites that redirects to other malicious files/urls. Everytime I check a website from the MDL list I found a new one.

It will be useful for you to monitor them in an automated way. Now that you have clean-mx with a great platform it's time to create yours.  ;D

jsunpack is available to download if you want to analyze javascript, pdf files etc.. The script is in python (I'm not familiar with this language so I can't help with this)

Wepawet can be used to send urls, pdf, flash, executable and we can retreive VirusTotal (% and virus names), Anubis (traffic capture and payload behaviors), ThreatExpert, Sunbelt, Prevx and tons of useful information.

I can also provide some scripts to check for redirections (headers redirects, javascript etc.)
Title: Re: Malware monitoring platform
Post by: SysAdMini on September 07, 2009, 09:51:53 am
I'm  very sceptical about automated malware monitoring.

In my opionion there is no reliable way to determine automatically if an url is malicious or not.
There are too many dependencies. Think about the Fragus kit where you first have to visit the show.php before you get access to the payload.
Other sites require special useragents or are only available to special geographic regions.
Some sites return http code 404, but deliver exploits anyway.

There are too many things to consider. Therefore you can't just pull all urls from db and try to download the content.
This is the reason why I don't do it for MDL. It has the disadvantage that a lot of MDL urls are already inactive or cleaned, but still listed as active.

I appreciate your idea in general, but I'm sceptical. And there is another drawback. You might run into legal issues if you flag sites as malicious mistakenly.
This is the reason why I check all urls manually before I add it to list and why I don't simply import submissions.
And even you do it this way, you can run into problems when you call a software "Rogue" if it is Adware. I have experienced too many of those cases.
Title: Re: Malware monitoring platform
Post by: Malware-Web-Threats on September 07, 2009, 10:04:48 am
Quote
In my opionion there is no reliable way to determine automatically if an url is malicious or not.
You have not totally wrong :-o

Quote
Other sites require special useragents or are only available to special geographic regions.
I use a US, UK IPs, some double check with Google, Yahoo and Bing as referer and have no problem with this.

Quote
Some sites return http code 404, but deliver exploits anyway
Most of them is because you must be redirect from another site to sucessfully load the exploit (referer needed)

Quote
You might run into legal issues
Not if you protect yourself with Terms and Conditions. Read the disclaimer for example: www.malwarepatrol.com

-------

And what about Threat Expert? (advanced automated threat analysis system)
Title: Re: Malware monitoring platform
Post by: Orac on September 07, 2009, 10:12:59 am
I agree with SysAdMini. There are way to many variables involved, until true AI becomes a reality i dont believe it will be possible to achive an effective, reliable automated method that can come close to matching, let alone beating the old mark one eyeball for the job.

Ive been involved with two attempts to automate the process, both ended up in dismal failure, dispite one of them being lead by someone who had done a similar think in realtion to phising sites which worked vey effectively.

I look forward one day to being proved wrong, until then its the old fashioned way iam afraid.
Title: Re: Malware monitoring platform
Post by: Malware-Web-Threats on September 07, 2009, 10:23:06 am
Quote
There are way to many variables involved

Yes may be but is this for the 10% of sites that you can't load in an automated way that you abandon the idea?

You must rethink - just take a look:

malwareurl.com

Most of them are added in an automated way.
Title: Re: Malware monitoring platform
Post by: cleanmx on September 07, 2009, 10:33:42 am
hi @all

I agree not all urls may be added automagically...
but as malwareurl do, so dow we at cleanmx.

we analyse threatexpert an anubis, we analyse links inside retrieved url's and we do some other vodoo....

I suppose less than 5% will be left over to be reviewed by human eye.

and legal.... i have no problems, I only collect these evil, and complain about them, i will never block someone....

I pers. think we should concentrate us on these 5% and we shall integrate our databases .... just in the way I do it for now on clean-mx !

malwaredomainlist=sub4 in my database
malwareurl=sub6

I wrote a request to http://www.malwarepatrol.net/ for dataexchange.
and I will reactive google.... theay stoped after a couple of transmissions...

-- gerhard
Title: Re: Malware monitoring platform
Post by: SysAdMini on September 07, 2009, 10:47:36 am
and legal.... i have no problems, I only collect these evil, and complain about them, i will never block someone....

It seems that you haven't got letters from lawyers.
Title: Re: Malware monitoring platform
Post by: cleanmx on September 07, 2009, 10:49:00 am
Quote
It seems that you haven't got letters from lawyers.

no never since 2004 !
Title: Re: Malware monitoring platform
Post by: RS-232 on September 07, 2009, 11:29:20 am
Quote
And there is another drawback. You might run into legal issues if you flag sites as malicious mistakenly.
....
And even you do it this way, you can run into problems when you call a software "Rogue" if it is Adware.
I have experienced too many of those cases.
...and as far as I can remember,JohnC also had ran into numerous such cases of complaints and legal threats...

Quote
Not if you protect yourself with Terms and Conditions. Read the disclaimer for example: www.malwarepatrol.com
From their FAQ:
Quote
# Can I get an unsanitized list of URLs?
We do not make unsanitized URLs public. If you have a real need for it, please contact us.
We exchange such lists with CSIRTs and known security groups.
...if the goal is to provide blocklists without full url links but just with the domain names,
then there are quite a few projects already doing so succesfully...why re-invent the wheel here?...

Still though,i agree that some kind of semi-automation/synchronization or even integration would be nice,
if it was to take place between the databases,ie.MDL,hpHosts,CleanMX etc etc...
But who has the knowledge/patience to code that and how,that certainly needs quite a lot of conversation...
Title: Re: Malware monitoring platform
Post by: SysAdMini on September 07, 2009, 11:34:40 am
Quote
And there is another drawback. You might run into legal issues if you flag sites as malicious mistakenly.
....
And even you do it this way, you can run into problems when you call a software "Rogue" if it is Adware.
I have experienced too many of those cases.
...and as far as I can remember,JohnC also had ran into numerous such cases of complaints and legal threats...


Oh yes, I can confirm that.
Title: Re: Malware monitoring platform
Post by: Malware-Web-Threats on September 07, 2009, 11:39:58 am
I already have all the stuff to do this.

Do you have tools if I send you a list of 1.000 exploits domains?

This can be done in an automated way using wepawet for example then based on a score we can add them on a private list for analysis (if exploits found, if urls leads to exe or pdf, if exe is detected with more than 4/40 on VT etc..)
Title: Re: Malware monitoring platform
Post by: cleanmx on September 07, 2009, 11:51:56 am
shure ... i have tools... knife... hammer...

-- gerhard

update this was inteded to be a little joke  ;)
Title: Re: Malware monitoring platform
Post by: sparsha on September 07, 2009, 12:53:54 pm
Anthony

Your idea will be excellent for the rotators!
Title: Re: Malware monitoring platform
Post by: Malware-Web-Threats on September 07, 2009, 03:26:16 pm
what do you mean by "rotators"?
Title: Re: Malware monitoring platform
Post by: cleanmx on September 07, 2009, 07:12:01 pm
hi Anthony,

I suppose he meant those  using (fast)-flux technologie (rotating ip's nameserver's etc...)

-- gerhard
Title: Re: Malware monitoring platform
Post by: SysAdMini on September 07, 2009, 07:17:42 pm
what do you mean by "rotators"?

http://www.avertlabs.com/research/blog/index.php/2009/01/05/inside-the-malicious-traffic-business/
Title: Re: Malware monitoring platform
Post by: sparsha on September 08, 2009, 12:01:39 pm
Anthony sorry for the confusion, SysAdMini picked up what's their on my mind  :)
Title: Re: Malware monitoring platform
Post by: Malware-Web-Threats on September 10, 2009, 06:33:10 pm
shure ... i have tools... knife... hammer...

-- gerhard

update this was inteded to be a little joke  ;)

you've forgot the bulletproof jacket.

I'm aware of these TDS servers (traffic direction system)

http://www.malwareurl.com/search.php?domain=&s=redirects&match=0&rp=50&urls=on&redirs=on&ip=on&reverse=on&as=on
http://www.malwareurl.com/search.php?domain=&s=in.cgi&match=0&rp=50&urls=on&redirs=on&ip=on&reverse=on&as=on
http://www.malwareurl.com/search.php?domain=&s=go.php&match=0&rp=50&urls=on&redirs=on&ip=on&reverse=on&as=on