Cyber Security Expo
 
Log Agent, log file recollection tool by Floydman on 06/09/02

As Presented to Securidad en Computo 2000, November 28th, 2000, in Mexico City
http://www.seguridad2000.unam.mx


You can distribute this document freely, as long as no changes are made to the file, or as long as credit for it is not pretended by someone else. All comments and suggestions about the material presented here should be directed at floydian_99@yahoo.com. If future versions of this document include add-ons coming from other people than me, then proper credit to the various authors will be clearly identified. All version updates of this document are to be released by me.

You can find this paper and more online at http://www.geocities.com/floydian_99/

Abstract

The goal of this paper is to present a tool made in Perl for recollecting log files from various applications and various machines into a central location.

Preface

When comes the time to choose computer security tools, one most wanted feature is the ability to centralize the information contained in the log files. Also, this prevents the evidence from being tampered by a potential intruder. So because of this, somewhat good products are overlooked because they fail to provide this single feature, and sometimes this leads to purchasing a product that offers (and sells) many features not necessarily needed, or products that are not as flexible as desired when comes the time to make it work on your environment. In order to resolve this, I programmed LogAgent 1.0, which is an agent that you can run on all your Windows NT machines to monitor the log files of various unrelated applications and to redirect any new input made to these files to a central location.

Targeted audience

This document is presented to anyone who has interests in computer security, NT Administration, computer monitoring, intrusion detection, Perl programming and computing in general.

Table of contents

1. About LogAgent
2. The source code
3. The experiment
4. Conclusion

1. About LogAgent

First of all, I"d like to thank Amine Moulay Ramdane, who programmed the AdvNotify PPM package. AdvNotify provides the functions needed to monitor directories and report when changes are made to these directories. I practically used the example file she provides with it and made some modifications in order to capture data and redirect it. Here"s how LogAgent works:

@mondir is the list of directories containing the log files that you want to monitor. If one folder contains more than one log file, it is not necessary to specify each one of them, specifying the folder will catch them all.

$mainlog is the central destination where you want to redirect all your log files, in UNC format.

We also determine the machine host name, IP address and the username logged on the machine, and we store these values in variables.

Then we start one thread for each element contained in the list @mondir. After the thread is started, we have to enable each one of them with EnableWatch(). Then, the program goes into an infinite loop in which it waits for changes to happen in the folders it monitors. When data is added to a log file, it is also captured by LogAgent, the host information is appended to the data, and then it sends it back to the central log file defined by $mainlog. If we get out of the loop (with [CRTL-C]), then we terminate all the threads we previously created.

In the likely event that you don"t want to install Perl on all your networked PCs, you can use perl2exe.exe to convert your Perl scripts to an executable .EXE file. With the full version, you can even add -gui option in order to prevent the console from showing, thus making LogAgent run in the background without being seen by the user.

ActivePerl can be downloaded from www.activestate.com
Perl2exe (demo) can be downloaded from www.indigostar.com

2. The source code

#!C:\perl
# Log Agent version 1.0 (c)2000
# By Floydman
# floydian_99@yahoo.com
# You can freely use and distribute this Perl program, and change the code to fit your purposes as well.
# If you feel that somehow you can improve this program, please let me know so I can update the original.
# I guess you can consider this is Open Source software.

# Using Win32::AdvNotify
# By Amine Moulay Ramdane
# Website: http://www.generation.net/~aminer/Perl/
use Win32::AdvNotify qw(FILE_NAME SIZE INFINITE Yes No
All %ActionName %ActionColor);
use Socket;
use Sys::Hostname;

# Define username, IP address and hostname of the local machine
my $host = hostname();
my $addr = inet_ntoa(scalar(gethostbyname($name)) || "localhost");
my $login = getlogin || getpwuid($<) || "not logged";

# @mondir contains the list of directories to be monitored
@mondir = ("C:/Program Files/Security/Grisoft/Avg6/Log/",
"C:/Program Files/Utils/GetRight/Log/",
"C:/Winnt/Internet Logs/",
"C:/Program Files/Winetd/logs/");

# $mainlog contains the location of the master directory
$mainlog = "//darkside/log$/";

my $obj = new Win32::AdvNotify()|| die "Can"t create object\n";

# Note: you launch a thread for each directory you want to monitor,
# meaning one thread per object in @mondir

print "Log Agent 1.0 by Floydman\n";

my $Thr1 = $obj->StartThread(Directory => $mondir[0],
Filter => All ,
WatchSubtree => No ) || die "Can"t start thread\n";

my $Thr2 = $obj->StartThread(Directory => $mondir[1],
Filter => All ,
WatchSubtree => No ) || die "Can"t start thread\n";

my $Thr3 = $obj->StartThread(Directory => $mondir[2],
Filter => All ,
WatchSubtree => No ) || die "Can"t start thread\n";

my $Thr4 = $obj->StartThread(Directory => $mondir[3],
Filter => All ,
WatchSubtree => No ) || die "Can"t start thread\n";

print "\nThreads started successfuly ...\n";

#DO NOT FORGET TO ENABLE EACH ONE OF YOUR THREADS
$Thr1->EnableWatch() || die "Problem starting EnableWatch()\n";
$Thr2->EnableWatch() || die "Problem starting EnableWatch()\n";
$Thr3->EnableWatch() || die "Problem starting EnableWatch()\n";
$Thr4->EnableWatch() || die "Problem starting EnableWatch()\n";

print "\nTo exit from the loop press [CTRL-C] ...\n\n";

while($Thr1->Wait(INFINITE))# exit with [Ctrl-C] signal
{
while($Thr1->Read(\@data))# exit when the list is empty
{for($i=0;$i<=$#data;$i++)
{$obj->TextColor($ActionColor{$data[$i]->{Action}});
open (LOGFILE, $data[$i]->{Directory}.$data[$i]->{FileName}) or die "Can"t open log file";
flock (LOGFILE, 1) or die "Can"t lock file";
@Lines = ;
open (MAINLOG, ">>".$mainlog.$data[$i]->{FileName}) or die "Can"t open master log file";
flock (MAINLOG, 2) or die "Can"t lock file for writing";
print MAINLOG $host," ", $addr," ", $login," ", $Lines[-1] or die "Can"t read from file";
close (MAINLOG) or die "Can"t close master log file";
close (LOGFILE) or die "Can"t close file";

}}
}

# never forget to terminate your threads.

$Thr1->Terminate(); # terminate thread1
$Thr2->Terminate(); # terminate thread2
$Thr3->Terminate(); # terminate thread3
$Thr4->Terminate(); # terminate thread4

# never forget to call the destructor

undef $obj;

3. The experiment

What we just saw is the source code as I used it to perform an experiment to verify if LogAgent could deliver its promises (note: this test was performed before I added the variables containing the host information). So I decided to monitor the log files from 4 different, unrelated applications, and forward any input made to these files to a central log file directory located on UNC share name \\DARKSIDE\Log$.

The applications monitored were Grisoft AVG AntiVirus, ZoneLabs ZoneAlarm, Headlight Software GetRight, and COTSE Winetd. I don"t see how monitoring GetRight log files could be useful in a security context, but I needed a good test bed, and GetRight spans more log than the other applications, so this is why we see it here. The Log$ share was empty prior to the beginning of the test. I copied a shortcut pointing to LogAgent.pl in the startup folder, and rebooted in order to get a fresh user session. Then I proceeded to generate activity that would be logged by the mentioned applications. After I finished generating log activity from these applications, I proceeded to collect the results. Here"s what was in the central Log$ directory:

D:\Log>dir
Volume in drive D is D
Volume Serial Number is 0840-C01D

Directory of D:\Log

09/25/00 08:31p

.
09/25/00 08:31p ..
09/25/00 08:31p 373 bind.log
09/25/00 07:44p 2,483 getright.log
09/25/00 07:38p 8,768 IAMDB.RDB
09/25/00 08:31p 41 restart.log
09/25/00 08:31p 36 shutdown.log
09/25/00 08:31p 104 startup.log
09/25/00 07:38p 366 ZALog.txt
9 File(s) 12,171 bytes
902,415,872 bytes free

bind.log, restart.log, shutdown.log and startup.log all belongs to Winetd, getright.log belongs of course to GetRight, and ZALog.txt belongs to ZoneAlarm. IAMDB.RDB also belongs to ZoneAlarm, but I don"t know exactly what is its purpose. Since it is not relevant for our analysis (because it contains no useful data), I simply disregard it. Grisoft antivirus didn"t detect any virus during the testing period, so it did not generate any log. Here"s the content of the other files:

bind.log

bind() listening to port 3 - 18:53:18 - 09/25/2000
bind() listening to port 13 - 18:53:18 - 09/25/2000
bind() listening to port 23 - 18:53:18 - 09/25/2000
bind() listening to port 25 - 18:53:18 - 09/25/2000
bind() listening to port 137 - 18:53:18 - 09/25/2000
bind() listening to port 139 - 18:53:18 - 09/25/2000
bind() listening to port 139 - 18:53:18 - 09/25/2000

getright.log

2000/09/25-19:04:36: File: D:\downloads\Hack\telnet server\fd-update.zip.GetRight = http://rapidus.tucows.com/files/fd-update.zip
2000/09/25-19:04:39: (Re)Started download: http://rapidus.tucows.com/files/fd-update.zip
2000/09/25-19:04:40: Could not resume (restarting from 0): http://rapidus.tucows.com/files/fd-update.zip
2000/09/25-19:04:58: (Re)Started download: http://rapidus.tucows.com/files/fd-update.zip
2000/09/25-19:04:58: Could not resume (restarting from 0): http://rapidus.tucows.com/files/fd-update.zip
2000/09/25-19:05:16: File: D:\downloads\Hack\telnet server\fd-update.zip.GetRight = http://rapidus.tucows.com/files/fd-update.zip
2000/09/25-19:05:16: (Re)Started download: http://rapidus.tucows.com/files/fd-update.zip
2000/09/25-19:05:17: Could not resume (restarting from 0): http://rapidus.tucows.com/files/fd-update.zip
2000/09/25-19:09:20: File: D:\downloads\Hack\proxy\CProxy.zip.GetRight = http://rapidus.tucows.com/files2/CProxy.zip
2000/09/25-19:09:21: (Re)Started download: http://rapidus.tucows.com/files2/CProxy.zip
2000/09/25-19:09:22: Resumed: http://rapidus.tucows.com/files2/CProxy.zip at: 0
2000/09/25-19:10:28: File: D:\downloads\Hack\proxy\as-setup.exe.GetRight = http://rapidus.tucows.com/files2/as-setup.exe
2000/09/25-19:10:28: (Re)Started download: http://rapidus.tucows.com/files2/as-setup.exe
2000/09/25-19:10:33: Resumed: http://rapidus.tucows.com/files2/as-setup.exe at: 0
2000/09/25-19:13:23: File: D:\downloads\Hack\telnet server\fd-update.zip.GetRight = http://cny.tucows.com/files/fd-update.zip
2000/09/25-19:13:27: (Re)Started download: http://cny.tucows.com/files/fd-update.zip
2000/09/25-19:13:27: Could not resume (restarting from 0): http://cny.tucows.com/files/fd-update.zip
2000/09/25-19:13:33: (Re)Started download: http://rapidus.tucows.com/files2/CProxy.zip
2000/09/25-19:13:35: Resumed: http://rapidus.tucows.com/files2/CProxy.zip at: 519767
2000/09/25-19:13:35: (Re)Started download: http://rapidus.tucows.com/files2/as-setup.exe
2000/09/25-19:13:40: Resumed: http://rapidus.tucows.com/files2/as-setup.exe at: 49638
2000/09/25-19:30:36: Finished downloading: http://rapidus.tucows.com/files2/CProxy.zip
2000/09/25-19:31:50: (Re)Started download: http://rapidus.tucows.com/files2/as-setup.exe
2000/09/25-19:31:51: Resumed: http://rapidus.tucows.com/files2/as-setup.exe at: 49638
2000/09/25-19:44:25: Finished downloading: http://rapidus.tucows.com/files2/as-setup.exe

restart.log

server restart- - 10:34:15 - 09/25/2000

shutdown.log

shutdown - - 10:29:40 - 09/25/2000

startup.log

Starting Server: darkside - 18:53:17 - 09/25/2000
Starting Server: darkside - 18:53:17 - 09/25/2000

ZALog.txt

PE,2000/09/25,18:53:00 -5:00 GMT,ZoneAlarm Internet Security Utility,203.110.251.2:53,N/A
FWIN,2000/09/25,18:57:00 -5:00 GMT,65.229.239.88:1901,65.227.240.78:139,TCP
FWIN,2000/09/25,19:19:22 -5:00 GMT,65.228.69.89:2936,65.227.240.78:139,TCP
PE,2000/09/25,19:37:49 -5:00 GMT,Xnews,203.110.251.2:53,N/A
PE,2000/09/25,19:38:06 -5:00 GMT,Xnews,203.110.251.2:53,N/A

For the sake of testing, here is what a log file looks like once I put in place the host information variables (using a new ZALog.txt):

ZALog.txt

test_host 10.1.4.87 Administrator FWIN,2000/11/14,15:50:08 -5:00 GMT,10.1.4.68:1676,10.1.4.87:4,TCP
test_host 10.1.4.87 Administrator FWIN,2000/11/14,15:50:08 -5:00 GMT,10.1.4.68:1676,10.1.4.87:4,TCP
test_host 10.1.4.87 Administrator FWIN,2000/11/14,15:50:08 -5:00 GMT,10.1.4.68:1676,10.1.4.87:4,TCP
test_host 10.1.4.87 Administrator FWIN,2000/11/14,15:50:08 -5:00 GMT,10.1.4.68:1676,10.1.4.87:4,TCP
test_host 10.1.4.87 Administrator FWIN,2000/11/14,15:50:08 -5:00 GMT,10.1.4.68:1676,10.1.4.87:4,TCP
test_host 10.1.4.87 Administrator FWIN,2000/11/14,15:50:08 -5:00 GMT,10.1.4.68:1676,10.1.4.87:4,TCP
test_host 10.1.4.87 Administrator FWIN,2000/11/14,15:50:08 -5:00 GMT,10.1.4.68:1676,10.1.4.87:4,TCP
test_host 10.1.4.87 Administrator FWIN,2000/11/14,15:50:54 -5:00 GMT,10.1.4.68:1683,10.1.4.87:65535,TCP

We can see that the IP address (10.1.4.87) is written twice, but with an application that fails to give such information, it does get really helpful in identifying the origin of a log entry.

4. Conclusion

LogAgent provides a very inexpensive way to track down all loggable activity on your Windows NT network. In fact, all you may have to pay is a license for Perl2exe full version, and this will provide you with the ability to make LogAgent (or any other of your Perl scripts) run as a background task without any visual indicator of their presence (except maybe for the Task Manager). With this simple but useful tool, you can now look at any security application that you may want to deploy, even if they don"t provide the ability to log centrally. LogAgent will take care of this task for you. Unfortunately, AdvNotify will only work on NT, but I am working to see if I can make a LogAgent that could work also on Windows 9x machines as well. Your input on this project is welcome.

Rate this article

All images, content & text (unless other ownership applies) are © copyrighted 2000 -  , Infosecwriters.com. All rights reserved. Comments are property of the respective posters.