Author Topic: Proxy Logs Analysis  (Read 5953 times)

0 Members and 1 Guest are viewing this topic.

June 15, 2009, 09:17:27 pm
Read 5953 times

Red

  • Newbie

  • Offline
  • *

  • 3
This is a bit of a tough question.

I'm currently grepping through a lot of proxy logs for particular malware related strings. The proxy logs are currently compressed in gzip files and using the command line to analyze through proxy logs.

gzip.exe -v -c -d C:\logs\regex.gz | perl ack.pl -i -h "stringtosearchfor" > output.txt

Ultimately, I'm researching a way to just take an entire input.txt file to grep with. I've found other means using BASH. I'm also restricted to the Windows CLI at the moment. There is also another option of using Cygwin. Although, I'm not sure how the formatting should be. I'm also concerned about performance since I'd like to grep for about 100-500 lines of entries.

Does anyone have a suggestion?

THANKS!

June 24, 2009, 11:25:13 pm
Reply #1

Shawn Jefferson

  • Newbie

  • Offline
  • *

  • 8
Don't know if this will work for you or not, but there are windows versions of grep available, and grep allows you to specify a text file of search terms.

June 26, 2009, 01:28:09 am
Reply #2

RS-232

  • Special Access
  • Sr. Member

  • Offline
  • *

  • 165
Kinda late reply,but anyway...
This one might be as well of interest to you,since you referred specifically to Windows systems:
http://www.microsoft.com/downloads/details.aspx?FamilyID=890cd06b-abf8-4c25-91b2-f8d975cf8c07&displaylang=en
Only for the "fun" of it...rs-232 aka sowhat-x aka younameit ;-)
http://www.youtube.com/watch?v=fADjY97_KTw

July 02, 2009, 04:03:35 pm
Reply #3

h4h4h4h4

  • Jr. Member

  • Offline
  • **

  • 11
This is a bit of a tough question.

I'm currently grepping through a lot of proxy logs for particular malware related strings. The proxy logs are currently compressed in gzip files and using the command line to analyze through proxy logs.

gzip.exe -v -c -d C:\logs\regex.gz | perl ack.pl -i -h "stringtosearchfor" > output.txt

Ultimately, I'm researching a way to just take an entire input.txt file to grep with. I've found other means using BASH. I'm also restricted to the Windows CLI at the moment. There is also another option of using Cygwin. Although, I'm not sure how the formatting should be. I'm also concerned about performance since I'd like to grep for about 100-500 lines of entries.

Does anyone have a suggestion?

THANKS!


The proxy logs i use are bziped.

make an input file called input.txt and inside of it put what you want to search.

bzgrep -f input.txt logstosearchagainst | tee output.txt

July 14, 2009, 07:59:45 pm
Reply #4

Red

  • Newbie

  • Offline
  • *

  • 3
Don't know if this will work for you or not, but there are windows versions of grep available, and grep allows you to specify a text file of search terms.

Yes, I'm already using gzip.exe which is a win32 binary.

July 14, 2009, 08:01:31 pm
Reply #5

Red

  • Newbie

  • Offline
  • *

  • 3

The proxy logs i use are bziped.

make an input file called input.txt and inside of it put what you want to search.

bzgrep -f input.txt logstosearchagainst | tee output.txt

This might be of some use, I'll give it a shot.

Thanks to all who replied. This put me in the right direction.