Post new topic Reply to topic  [ 73 posts ]  Go to page 1, 2  Next

ClamD Interface like the one for SpamD (network)
Yes 86%  86%  [ 24 ]
No 14%  14%  [ 4 ]
Total votes : 28
Author Message
 Post subject: Interface to ClamD just like the one for SpamD
PostPosted: 2009-04-29 12:58 
Normal user

Joined: 2009-02-24 11:17
Posts: 31
Location: Leicester, UK
Whilst clamwin seems to play nicer on win32 than sawin32 does (just my opinion), it would be good to be able to have an option to use clamd running on another machine, either real or virtual like can be done with spamassassin (spamd). I have spamd running on a Linux virtual machine with better perfomance than running sawin32 locally. I would like to run clamd in the same way, either on the same machine or in its own virtual or real machine if required. (optionally you could just use clamd on windows which would do away with any virtual machines if they aint your cup of tea)


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2009-05-01 17:19 
Normal user

Joined: 2006-04-14 00:00
Posts: 163
Technically, you cannot do that the same way that it is done with SpamD. The client code for ClamAV passes the file path as a string to ClamD. It does not pass the entire file like SpamC does to SpamD. So you cannot offload ClamAV the same way to another machine. I wish you could, but you can't.

What you can do (what we already do) is run ClamD on the same Linux boxes as SpamD and set up a spam rule for running ClamAV. Any message that triggers the ClamAV "spam rule" gets 200 points added to its SA score. And ClamD runs on the Linux based SpamD boxes, so all of the CPU usage of SpamD and ClamD is offloaded to the Linux boxes. This is the only solution I have found so far for offloading ClamAV.


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2009-05-01 17:25 
Normal user

Joined: 2006-04-14 00:00
Posts: 163
If you are interested in setting up ClamAV as a SA rule, check out this Wiki article:
http://wiki.apache.org/spamassassin/ClamAVPlugin

Recommended changes:
In the sample clamav.cf file, change 10 to 200 if you want to have virus to get an SA score of 200 instead of just 10.


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2009-05-18 21:26 
Normal user

Joined: 2009-05-12 23:06
Posts: 52
Location: Denmark
Shiloh wrote:
Technically, you cannot do that the same way that it is done with SpamD. The client code for ClamAV passes the file path as a string to ClamD. It does not pass the entire file like SpamC does to SpamD. So you cannot offload ClamAV the same way to another machine. I wish you could, but you can't.


That is wrong, you can send files to a remote ClamD server over a network without the need to shared storeage.

My plugin call MSWclamDscan.exe that i have made for the Mailsweeper scanner software is sending files over a TCP connection to a remote Linux server with ClamD running.

http://www.tooms.dk/Tblog/Showblog1.asp ... 2053506046

http://www.tooms.dk/software/mswclamdscan/default.asp

The software is still beta code so only use it for testing.

_________________
Any comment or statements is my own and have no relationship to my workplace


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2009-05-23 18:19 
Normal user

Joined: 2006-04-14 00:00
Posts: 163
Tooms: I would love it if the core ClamAV development would absorb that code so this feature could be implemented. Unfortunately, the current ClamAV code (without this mod) does not allow for this. That would be a terrific improvement for ClamAV.

I have a quick question, though. Does the MSWclamDscan mod need to be implemented in both the client and server, or just the client? I have not used MSWclamDscan and I am assuming it would need to be done on both ends, but the instructions with MSWclamDscan seem to say that a standard installation of ClamD will work.


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2009-05-24 20:44 
Normal user

Joined: 2009-05-12 23:06
Posts: 52
Location: Denmark
Shiloh wrote:
Tooms: I would love it if the core ClamAV development would absorb that code so this feature could be implemented. Unfortunately, the current ClamAV code (without this mod) does not allow for this. That would be a terrific improvement for ClamAV.

I have a quick question, though. Does the MSWclamDscan mod need to be implemented in both the client and server, or just the client? I have not used MSWclamDscan and I am assuming it would need to be done on both ends, but the instructions with MSWclamDscan seem to say that a standard installation of ClamD will work.


The ClamD server support this as standard, you just have to enable it in the config whene you tell it to listens on a port.
I don't know if this work in every windows port of the ClamD, but i have tryed a number of them and there it is working.
I my docs with the MSWclamDscan app there is a list of know port that i know is working or has been reported by others to work.

The code side of it is much the code they use to day, instedt of tell the ClamD server where the file is on disk then you tell it to "stream" and it will return a port number where you send the file to, so there is no big magic in this. If you look into the MAN page of ClamD there you can see information about how to do this.


Until they include a native version of ClamDscan into the Hmailserver, then you can use my MSWclamDscan as commandline scanner.
My version of ClamDscan support multi ClamD server to make it more safe as if one ClamD server fail then the MSWclamDscan will reconnect to the next server and it use then as load share so if you have high mail load then just run more ClamD servers to share the load, MSWclamDscan also have number safty checks and timeout function to make sure the scanning of content is working.

Hope you can use it.

_________________
Any comment or statements is my own and have no relationship to my workplace


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-04 06:59 
Normal user
User avatar

Joined: 2005-07-20 17:08
Posts: 125
Location: Catawissa, PA
I would also really like if i could run clamAV on a separate Linux box!


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-04 16:31 
Senior user
Senior user
User avatar

Joined: 2009-04-17 18:10
Posts: 268
Location: The land of Beer and Sauerkraut!
PeterK2003 wrote:
I would also really like if i could run clamAV on a separate Linux box!

What stops you from doing it right now? ClamD doesn't have to run at the same machine, ClamDScan can connect to a remote ClamD daemon as well. If you have more than one ClamAV server it might make sense to run the above mentioned tool of Tooms instead of ClamDScan.

Best regards,

Nico


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-04 16:36 
Normal user
User avatar

Joined: 2005-07-20 17:08
Posts: 125
Location: Catawissa, PA
ohhh i didn't read it clearly....is there a "how to" post?


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-04 17:18 
Senior user
Senior user
User avatar

Joined: 2009-04-17 18:10
Posts: 268
Location: The land of Beer and Sauerkraut!
In case you're referring to how ClamDScan (or Tooms tool) can be implemented as external scanner in hMS I think there are several threads/posts here (I'd suggest a search). Just don't mix it up with ClamWin, that is something completely different.

Best regards,

Nico


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-04 21:19 
Normal user
User avatar

Joined: 2005-07-20 17:08
Posts: 125
Location: Catawissa, PA
I couldn't find a how-to but there is a lot of threads about clamAV and phpBB's search isn't the best so I may have just missed it.


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-05 01:53 
Senior user
Senior user
User avatar

Joined: 2009-04-17 18:10
Posts: 268
Location: The land of Beer and Sauerkraut!
horndog wrote:
I'm also curious if the "clamdscan" from ClamAV [tBB] has the necessary switches to send and receive using tcp/ip to another ClamAV LAN server?

There are no special switches for ClamDScan as it just reads the clamd.conf file. If you want ClamDScan to use a remote ClamD you have to adjust 'TCPSocket' and 'TCPAddr' in the local clamd.conf to reflect the IP/port used by the ClamD server and all should be fine.

Best regards,

Nico


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-05 10:07 
Senior user
Senior user
User avatar

Joined: 2009-04-17 18:10
Posts: 268
Location: The land of Beer and Sauerkraut!
Again, ClamD does NOT have to run on the same machine. It doesn't even need to be there at all, also of course databases don't have to be on the local machine if ClamD isn't running.

The stream protocol Win32 ClamD/ClamDScan uses is exactly the same as in the Linux version.

There are several third party signature update scripts for Linux, see http://www.sanesecurity.co.uk/download_ ... _linux.htm I'd recommend the one from Bill Landry.

Best regards,

Nico


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-05 23:25 
Normal user

Joined: 2009-05-12 23:06
Posts: 52
Location: Denmark
horndog wrote:
I downloaded the file (mswclamdscan.1.2009.11.09.2000_RC1.zip) from tooms web site:
http://www.tooms.dk
and read the doc file.
Quote:
How-to Install the MSWClamDscan plug-in for Mailsweeper 5.x Single server

This looks pretty slick. I'm just curious if it's ready for production? I'm also curious if the "clamdscan" from ClamAV [tBB] has the necessary switches to send and receive using tcp/ip to another ClamAV LAN server?


Please note that i have release a new version
http://www.tooms.dk/Tblog/Showblog1.asp ... 2306513681
http://www.tooms.dk/software/mswclamdscan/default.asp

hope you can use it.

_________________
Any comment or statements is my own and have no relationship to my workplace


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-05 23:39 
Normal user

Joined: 2009-05-12 23:06
Posts: 52
Location: Denmark
horndog wrote:
If the 'TCPSocket' and 'TCPAddr' are changed to direct to a different ClamAV server does the localhost clamD need to be running and data base need to be updated? Can the localhost 'clamDscan' successfully work with a LAN Linux install of ClamAV? Is there a third party signature program for ClamAV Linux? The Linux questions are for anyone that want to have a Linux SpamAssassin and ClamAV server.
Thanks for this great information!


"ClamD" is the server part that can be installed on a aother pc in the network or on the same server as hmailserver.
ClamD server can be the linux version on a linux server or it can be on windows where you can use Nico's windows port of ClamAV.
On the ClamD server you simple set the option "TCPsocket=3310" in the ClamD.conf config file and restart the server, so now it can be used over the network.
if you set the "TCPaddr" you will lock it to that interface and if you not set it then it will listen on all interfaces.

"ClamDscan" is the client there is using the "clamD" server over the network
When using the ClamDscan client you then have to tell it where the server is on the network.
My app MSWclamdscan is a clone of "clamDscan" but it is design to be used in highload mailscanner and can use multi clamd server to share the load and more safe so if one clamd server breck down or is updating it patten files then mswclamdscan just fail over to another clamd server with the MTA(hmailserver) ever know it.

_________________
Any comment or statements is my own and have no relationship to my workplace


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-05 23:50 
Normal user

Joined: 2009-05-12 23:06
Posts: 52
Location: Denmark
horndog wrote:
Hello all,
I'm missing something on how to implement HMS to use ClamAV using tcp/ip. What do you use in the Scanner executable field? How would do this when ClamAV is on a different network server? Do I need Tooms's tool on the HMS server to accomplish this?
Thanks


my tool MSWclamDscan is only the client part so you have to install a clamD server somewhere and that can be on linux or on windows with Nico's ClamAV.

if you use my MSWclamDscan tool then just follow the guide for installing and when it is working you can then setup hmailserver to use the MSWclamDscan as a antivirus scanner.

1.
go to: hmailserver gui->Settings->Antivirus->select tab "External virus scanner"

2.
check the "use external scanner"

3.
In the textbox "scanner executable" inset "d:\mswclamdscan\mswclamdscan.exe /mail:"%FILE%""
note the qwote around the %file% must be there as the mail path has space in it.

4.
set "Return value" to 1

5.
goto the Status page and restart the hmailserver to reload the new config.

6.
verify that it now detect virus mails by setting some virus test mails via hmailserver


I think this is it

_________________
Any comment or statements is my own and have no relationship to my workplace


Last edited by Tooms on 2010-01-06 10:18, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-06 10:10 
Normal user

Joined: 2009-05-12 23:06
Posts: 52
Location: Denmark
Webpage with virus found.

If you download my mswclamdscan and set it up then you in the config gui will see a part to setup a database for the logging and when the mswclamdscan detect a virus when scanning then it will read the config from the reg and if it is config to log the information then it will inset a entry to the database.

The database with the virus log is very simple and you can easy make your own page to display the information the way you like or you can use the sample page that i have include and it has full sourcecode.
To see a live demo of this page then goto http://www.tooms.dk/ and click on "E-mail Virus Stat" in the menu

For now the database can only be a Mircosoft database like msde, mssql 20xx

_________________
Any comment or statements is my own and have no relationship to my workplace


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-06 10:17 
Normal user

Joined: 2009-05-12 23:06
Posts: 52
Location: Denmark
horndog wrote:
horndog wrote:
Is this a ready made script/web page or do I have to incorporate the code into an existing web page?

What I meant to ask was: Can this be made in a non *.ASPX page such as php or HTML?


The data is simple stored in a standard MS database so you can make a page in any format you like as long as it can read the database and build a webpage.

I have include a .aspx webpage with full sourcecode in the mswclamdscan pack.

_________________
Any comment or statements is my own and have no relationship to my workplace


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-06 11:07 
Normal user

Joined: 2009-05-12 23:06
Posts: 52
Location: Denmark
horndog wrote:
Tooms wrote:
For now the database can only be a Mircosoft database like msde, mssql 20xx

My web site uses Apache, php, and MySQL on Windows and I have years invested in knowing and working with these applications. When you include code for MySQL I'll give the dynamic web page a try. I'll try mswclamdscan right away. Thanks


Some day i maybe will include mysql option if the request if high for it.
You know time is limit when it is ones freetime

_________________
Any comment or statements is my own and have no relationship to my workplace


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-08 11:09 
Normal user

Joined: 2009-05-12 23:06
Posts: 52
Location: Denmark
horndog wrote:
Tooms wrote:
3.
In the textbox "scanner executable" inset "d:\mswclamdscan\mswclamdscan.exe /mail:"%FILE%""
note the qwote around the %file% must be there as the mail path has space in it.


I got mswclamdscan working. Nice job! The only bug was your step three. It didn't work with the two sets of double quotes. but with "%FILE%" it worked well. I tested with the file in the folder "testmail_spam1.msg" detecting it with no problem. I had to change MAX STREAM to "0" and It even complained when freshclam was late to update. Next I'll add some more ClamAV servers. Thanks


It was not a two-set double quotes it was one double quotes around the %FILE% and another set around the whole commandline but i can see this was misleding and wrong, but good to hear that it is working.

the "testmail_spam1.msg" is only a sample spam mail and does not include a virus so i hope when say it "detecting it with no problem" you mean that it scan the mail file with on error and return exitcode 0 (no virus found)

MAXSTREAM, instedt of setting this to 0 then it is better to set a byte vaule there is a bit higher then mswclamdscan max filesize value is set to. This way it better protect the server.

I new to hmailserver so i hope you can help me.
the mswclamdscan plugin can return a logfile with one log line with the status and virus name, do you know how to return this information back to hmailserver and use it in hmailserver like in a reply mail to sender or/and as information to the mailadmin.

_________________
Any comment or statements is my own and have no relationship to my workplace


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-08 15:26 
Moderator
User avatar

Joined: 2007-06-14 05:12
Posts: 11891
Location: 'The Outback' Australia
Tooms wrote:
the mswclamdscan plugin can return a logfile with one log line with the status and virus name, do you know how to return this information back to hmailserver and use it in hmailserver like in a reply mail to sender or/and as information to the mailadmin.

http://www.hmailserver.com/documentatio ... _antivirus

_________________
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
Documentation


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-08 16:28 
Normal user

Joined: 2009-05-12 23:06
Posts: 52
Location: Denmark
mattg wrote:
Tooms wrote:
the mswclamdscan plugin can return a logfile with one log line with the status and virus name, do you know how to return this information back to hmailserver and use it in hmailserver like in a reply mail to sender or/and as information to the mailadmin.

http://www.hmailserver.com/documentatio ... _antivirus



Yes thanks that is a good answar :lol:

I hade hope there was a way to return the Virus name for logging information, but maybe this will come some day.

_________________
Any comment or statements is my own and have no relationship to my workplace


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-09 05:16 
Moderator
User avatar

Joined: 2007-06-14 05:12
Posts: 11891
Location: 'The Outback' Australia
Tooms wrote:
I hade hope there was a way to return the Virus name for logging information, but maybe this will come some day.

Your antivirus may log the virus name. Mine certainly does.

_________________
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
Documentation


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-09 13:02 
Senior user
Senior user
User avatar

Joined: 2009-04-17 18:10
Posts: 268
Location: The land of Beer and Sauerkraut!
I think what he was asking for is if there's a way to pass the name of a found malware back to hMS so that the sender or admin can be notified.

Best regards

Nico


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-09 15:31 
Normal user

Joined: 2009-05-12 23:06
Posts: 52
Location: Denmark
tBB wrote:
I think what he was asking for is if there's a way to pass the name of a found malware back to hMS so that the sender or admin can be notified.

Best regards

Nico


Yes that what i was looking for

_________________
Any comment or statements is my own and have no relationship to my workplace


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-10 06:27 
Moderator
User avatar

Joined: 2007-06-14 05:12
Posts: 11891
Location: 'The Outback' Australia
Tooms wrote:
tBB wrote:
I think what he was asking for is if there's a way to pass the name of a found malware back to hMS so that the sender or admin can be notified.

Best regards

Nico


Yes that what i was looking for

That's what I expected.

I believe that this is NOT currently possible.

My Antivirus (Avast Corporate, via ADNM) sends me an e-mail every defined period (in my case weekly) detailing which computer, which virus, how many etc . All complete with graphs and my company letterhead.

You could parse your antivirus logs and create such a report. I do similar things with other log files, for instance I track RDP logons by username, originating IP address etc from the IIS logs.

Matt

_________________
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
Documentation


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-10 10:41 
Normal user

Joined: 2009-05-12 23:06
Posts: 52
Location: Denmark
mattg wrote:
Tooms wrote:
tBB wrote:
I think what he was asking for is if there's a way to pass the name of a found malware back to hMS so that the sender or admin can be notified.

Best regards

Nico


Yes that what i was looking for

That's what I expected.

I don't believe that this is NOT currently possible.

Matt



Thanks

_________________
Any comment or statements is my own and have no relationship to my workplace


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-10 11:32 
Senior user
Senior user
User avatar

Joined: 2009-04-17 18:10
Posts: 268
Location: The land of Beer and Sauerkraut!
horndog wrote:
EDIT: The name of the virus is not passed on. I think that would be near impossible at this point. ...And I don't think I would want that information passed on to the user.

mattg wrote:
My Antivirus (Avast Corporate, via ADNM) sends me an e-mail every defined period (in my case weekly) detailing which computer, which virus, how many etc . All complete with graphs and my company letterhead.

You could parse your antivirus logs and create such a report. I do similar things with other log files, for instance I track RDP logons by username, originating IP address etc from the IIS logs.

If external tools are allowed to write to the mail a X-ClamAV header could be added which contains the virus name. Based on this header further filtering/processing should be easily possible. IMO sending bounces to the outside is a bad idea but local users should be provided with all necessary informations in realtime (including the virus name).

Best regards,

Nico


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-10 14:43 
Moderator
User avatar

Joined: 2007-06-14 05:12
Posts: 11891
Location: 'The Outback' Australia
Tooms wrote:
mattg wrote:
I don't believe that this is NOT currently possible.

Thanks

Sorry, my bad English (and my only language at that)

I meant to say 'I believe that this is NOT possible' or 'I DON'T believe that this is possible', ie you can't do it.

You can not pass the virus name directly back to hMailserver from Clam.
You can to to headers as tBB suggest though.

AVAST also adds headers 'X-Antivirus' which is Avast + current virus database number + 'inbound' vs 'outbound' and 'X-Antivirus-Status' which is 'clean' or 'infected + virus name.

_________________
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
Documentation


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-10 16:39 
Senior user
Senior user
User avatar

Joined: 2009-04-17 18:10
Posts: 268
Location: The land of Beer and Sauerkraut!
mattg wrote:
AVAST also adds headers 'X-Antivirus' which is Avast + current virus database number + 'inbound' vs 'outbound' and 'X-Antivirus-Status' which is 'clean' or 'infected + virus name.

I guess this is something different. I don't use Avast but presumably it either injects itself into the LSP stack or it installs a hidden proxy over which all traffic is routed but both methods are applied before hMS even sees the mail (or in case port 110 is monitored after hMS processed the mail) so I'm still not sure if external processes are allowed to write directly to the mail when it's already in hMailServer's queue. Unfortunately I don't use hMS myself in production yet and have only played around with it so I can't tell much but perhaps someone can clarify *coughmartincough* ;)

If external processes are able to add headers to the mail then it wouldn't be much of a problem to parse the virus name out of ClamDScan's output (I wouldn't try to parse the log) and add a header. It could be even done by a simple batch file which is called instead of ClamDScan and a external tool which adds the header. I could even provide such thing, should anyone be interested.

Best regards,

Nico


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-10 23:43 
Normal user

Joined: 2009-05-12 23:06
Posts: 52
Location: Denmark
horndog wrote:
horndog wrote:
Tooms wrote:
"ClamDscan" is the client there is using the "clamD" server over the network
When using the ClamDscan client you then have to tell it where the server is on the network.
My app MSWclamdscan is a clone of "clamDscan" but it is design to be used in highload mailscanner and can use multi clamd server to share the load and more safe so if one clamd server breck down or is updating it patten files then mswclamdscan just fail over to another clamd server with the MTA(hmailserver) ever know it.

My question for Tooms is load sharing. Can, and does MSWclamdscan stream to more than one clamd server at a time to share the load of a busy production HMS?

...And is there any way to do load sharing with your MSWspamdscan for SpamAssassin?

Question for Tooms.


As MSWclamdscan is simply a commandline tool there is reading the reg for config and two mswclamdscan can not interact.
So the "load sharing" is simply that it pick a random server from the server list and if that fails to give a good respone back then it will pick another server from the list until it get a good response or it hit the max retry limit, but all limits and timeout value can be config from the GUI


to see what happen then try setup 2 or more ClamD servers and add then in the MSWclamDscangui and then run this commandline a number of times..

c:\mswclamdscan>mswclamdscan.exe /mail:Name_Of_Some_mail.eml /debug /devdebug

note that it will at random pick a server to do some sort of load sharing.


Now try shutdown now of the ClamD servers on the list and run the commandline again.

c:\mswclamdscan>mswclamdscan.exe /mail:Name_Of_Some_mail.eml /debug /devdebug

Now note that when it try use the server that is down then it will timeout waiting for the server to response and then pick another.


I am playing with a monitor app for clamd that dynamic can enable/disable servers from the server list so the mswclamdscan not wasting time on the down server and the app will also alart the admin about there is a issue, but this app is long away.



Hoe this made it more clear.

_________________
Any comment or statements is my own and have no relationship to my workplace


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-10 23:48 
Normal user

Joined: 2009-05-12 23:06
Posts: 52
Location: Denmark
horndog wrote:
Tooms wrote:
"ClamDscan" is the client there is using the "clamD" server over the network
When using the ClamDscan client you then have to tell it where the server is on the network.
My app MSWclamdscan is a clone of "clamDscan" but it is design to be used in highload mailscanner and can use multi clamd server to share the load and more safe so if one clamd server breck down or is updating it patten files then mswclamdscan just fail over to another clamd server with the MTA(hmailserver) ever know it.

My question for Tooms is load sharing. Can, and does MSWclamdscan stream to more than one clamd server at a time to share the load of a busy production HMS?

...And is there any way to do load sharing with your MSWspamdscan for SpamAssassin?


My MSWspamC client is also supporting multi SpamD server and will readom pick one from the list to load share.

note if your running SpamD and/or ClamD on linux platform i don't think you will have any performance issue the load sharing is more for safty so if one server goes down then it will not halt the mail trafic.

_________________
Any comment or statements is my own and have no relationship to my workplace


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-11 00:53 
Senior user
Senior user
User avatar

Joined: 2009-04-17 18:10
Posts: 268
Location: The land of Beer and Sauerkraut!
horndog wrote:
Does MSWclamdscan have the ability to choose another ClamAV server when one is in used already by HMS but yet still gives a good response?
ClamD supports multiple threads so as long as the number of connections don't exceed the MaxThreads value configured in ClamD.conf it will always give a good response.

However, it might happen that ClamD is reloading it's databases if there was a update at the time a connection comes in and during this time it doesn't scan anything (of course). Depending on the hardware it might also take way more than 30 sec. to load and unpack databases, specially if there are a lot of third party signatures involved so if 30 sec. is the hardcoded timeout I would really suggest to make that configurable.

Best regards,

Nico


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-11 01:18 
Senior user
Senior user
User avatar

Joined: 2009-04-17 18:10
Posts: 268
Location: The land of Beer and Sauerkraut!
horndog wrote:
If clamAV was reloading it's databases at the time MSWclamdscan should roll over to another LAN clamav install. ( as per MSWclamdscan docs).

It does, but the upcoming hMS interaction with ClamAV probably won't and that's what I was referring to (and what this thread is about).
horndog wrote:
The hardcoded timeouts is in HMS and is not configurable by the user and with a busy production HMS I don't think that would be a wise move.

Do you think it's better if a mail isn't scanned at all? The definition of a loaded server depends pretty much on the hardware. A Pentium3-500 on a dialup line can be probably called 'under load' with 10 mails per minute while I call a load +10 mails per second. However, all that could happen is that hMS builds up a queue and I think there is no limit except the available space.

Therefore the processing timeout values of a mailserver should be IMO adaptable to the hardware/connection speed.

Best regards,

Nico


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-11 02:31 
Senior user
Senior user
User avatar

Joined: 2009-04-17 18:10
Posts: 268
Location: The land of Beer and Sauerkraut!
horndog wrote:
Your right about this being slightly of topic but seeing as how you and Tooms probably get email notifications a new thread would not be desirable at this time.

I guess Tooms and me would quickly monitor the new thread as well ;)

However, as Martin stated that the feature will be integrated this poll is probably useless anyway. We could start a new one about configurable timeout values though :lol:

Best regards,

Nico


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-11 18:02 
Senior user
Senior user
User avatar

Joined: 2009-04-17 18:10
Posts: 268
Location: The land of Beer and Sauerkraut!
horndog wrote:
This is good news! I hope this will speed up the process and have less time outs.

Hmmm..I fear this will have little to no effect on the timeouts because the calling of ClamDScan.exe doesn't cause that much CPU overhead. Even less if it's already in Win32's cache (which it most probably is because it's called quite often at a mailserver). What would help in this case is, well, a configurable timeout value :D
horndog wrote:
Seeing as how you brought up this idea, how about posting a new Feature request on "configurable timeout values." After all, you know how to get things done here! :)

Hehe. Well, I don't use hMS in production yet so IMO I shouldn't start a poll. Feel free to do so if you like :)

Best regards,

Nico


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-11 23:07 
Senior user
Senior user
User avatar

Joined: 2009-04-17 18:10
Posts: 268
Location: The land of Beer and Sauerkraut!
horndog wrote:
I just had a "Gregory House" moment! The biggest problem is ClamAV occasionally times out while reloading it's DB

Not sure what "Gregory House moment" means but let me quote myself:
tBB wrote:
However, it might happen that ClamD is reloading it's databases if there was a update at the time a connection comes in and during this time it doesn't scan anything (of course). Depending on the hardware it might also take way more than 30 sec. to load and unpack databases, specially if there are a lot of third party signatures involved so if 30 sec. is the hardcoded timeout I would really suggest to make that configurable.

So it's not ClamAV that times out, it's hMS!
horndog wrote:
So it appears that there are two main reasons for SpamAssassin to error all related to ClamAv.
Related to ClamAV? ClamAV has to load and unpack a huge database. If third party signatures are loaded which can be quite complex it takes even longer. Other virus scanners also don't scan anything during they reload their databases and queue the connections and other mailservers handle that as well.

Best regards,

Nico


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-12 02:33 
Senior user
Senior user
User avatar

Joined: 2009-04-17 18:10
Posts: 268
Location: The land of Beer and Sauerkraut!
horndog wrote:

Well, I know who Dr. House is but if you had googled for your own sentence http://www.google.com/search?hl=en&q="Gregory%20House%20moment" you would have found that there is exactly one hit which explains pretty much nothing.
horndog wrote:
HMS doesn't know, or care for that matter, what ClamAv has to do. When HMS calls ClamAV
to scan an email and when it does not get a response in a timely fashion it has a certain length of time to wait if that time is exceeded HMS times out, but who timed out first?

Simply: It's not ClamAV which timeouts because it has it's own definition of several types of timeouts (which are btw. configurable) and it clearly says in it's output/log/stream response if it timeouts. Otherwise, applications which call ClamAV have to wait until ClamAV has finished the scan, or until ClamAV indicates that it had a timeout, or it's simply not ClamAV's problem if it finishes the scan like you said in a "timely fashion" but as I have stated now numerous times, it's IMO impossible to give a general statement (or hardcoded value) about what a "timely fashion" is because it depends on your hardware. If you have a server with a quad core and a raid system below you won't even notice if ClamD reloads it's databases so, to copy your own statement:
horndog wrote:
It's not good to have an email server on a system that is not cutting edge if the traffic is high
So, according to yourself the hardware on which your ClamAV installation runs is apparently not fast enough for hMS and it's hardcoded timeout value :)

Best regards,

Nico


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-12 10:56 
Senior user
Senior user
User avatar

Joined: 2009-04-17 18:10
Posts: 268
Location: The land of Beer and Sauerkraut!
horndog wrote:
If your trying to win your case by using minutia, I find this tedious and a waste of time.
What I meant was that I had an epiphany.
Dude, not everyone on the planet is a fan of this TV series. I am clearly not and as such, I had no idea what you mean. However, as soon as someone starts to quote dictionaries and/or google, any discussion from this point on is usually moot anyway.
horndog wrote:
You clearly don't under my point! You might as will have a line in you config file called "Fried Green Tomatoes!" If ClamAV does not respond when called upon then as far as HMS is concerned ClamAV is DOA!
OK, I'll try to explain one last time: ClamAV is not unresponsive or hangs or whatever you try to emphasize. It queues the incoming connections until the reloading of the database is finished, then it scans them. Period.

Nico


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-12 11:30 
Normal user

Joined: 2009-05-12 23:06
Posts: 52
Location: Denmark
horndog wrote:
Tooms wrote:
So the "load sharing" is simply that it pick a random server from the server list and if that fails to give a good respone back then it will pick another server from the list until it get a good response or it hit the max retry limit, but all limits and timeout value can be config from the GUI

There is a 30.0 second time out hard coded in to HMS(I saw Martin post this same were on this forum). So every thing has to happen in that time frame including spam checks DNS blacklists and SURBL checks not to mention virus checks. So my point is time is at the up most importance. Does MSWclamdscan make the virus scan presses faster or slower?

Does MSWclamdscan have the ability to choose another ClamAV server when one is in used already by HMS but yet still gives a good response?


No the MSWclamdscan is design to be fast, small, simple and safe, so there is not sort of check of the server it will simple pick one from the list and try it and if it fail then it will try another if it is below the max_scan_time value.
So if your saying the max scan time allowed in HMS is 30 sec. then config mswclamdscan to use max scan time as 30 sec or less, but you have to check that the biggest file size you allow can be scan within the 30 sec.

If you have download my mswclamdscan pack then try run the config gui mswclamdscangui.exe and add your ClamD servers and then select one server and click on verify, it will now try to test the server and tell you how long time it use to scan the biggest file size allowed by the GUI


Quote:
Tooms wrote:
My MSWspamC client is also supporting multi SpamD server and will readom pick one from the list to load share.

Is there any way to integrate MSWspamC into HMS? HMS uses TCP/IP currently?
Would you consider writing a script to do this for the EventHandlers.vbs?

Thanks


How Martin is making the SpamC client and maybe the ClamDscan client in HMS is some thing he only can answer.

Horndag, I am still new to HMS and learning also i am only one privat person there is make my tools in my free time, so i can not keep up will all the requests ;-)

But i understand that you like to scan incoming mails and reject the bad ones before they get in, I think can easly be don as i have seen other scripts on the forum where there is run a .exe and then use it exitcode to do the action.

_________________
Any comment or statements is my own and have no relationship to my workplace


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-12 11:35 
Normal user

Joined: 2009-05-12 23:06
Posts: 52
Location: Denmark
horndog wrote:
tBB wrote:
However, it might happen that ClamD is reloading it's databases if there was a update at the time a connection comes in and during this time it doesn't scan anything (of course). Depending on the hardware it might also take way more than 30 sec. to load and unpack databases, specially if there are a lot of third party signatures involved so if 30 sec. is the hardcoded timeout I would really suggest to make that configurable.

If clamAV was reloading it's databases at the time MSWclamdscan should roll over to another LAN clamav install. ( as per MSWclamdscan docs).

The hardcoded timeouts is in HMS and is not configurable by the user and with a busy production HMS I don't think that would be a wise move.


I use mswclamdscan at work and we have to ClamD linux servers as backend and i have jet to see a fail mail do to database updating as the mswclamdscan simple retry until it is a good scan response or max scan time is up(can be config via gui)

from time to time i also rebooting and updating the ClamD linux servers and mail flow just goes on and there is no errors.

_________________
Any comment or statements is my own and have no relationship to my workplace


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-12 11:43 
Normal user

Joined: 2009-05-12 23:06
Posts: 52
Location: Denmark
horndog wrote:
tBB wrote:
It does, but the upcoming hMS interaction with ClamAV probably won't and that's what I was referring to (and what this thread is about).

Your right about this being slightly of topic but seeing as how you and Tooms probably get email notifications a new thread would not be desirable at this time.

I am only one human :) and will try answer what i can and yes maybe it is time to move this long talk to it own thread and not hijack this one as it seems more about my mswclamdscan then "Interface to ClamD just like the one for SpamD"

Quote:
tBB wrote:
Do you think it's better if a mail isn't scanned at all if the timeout value is exceeded? Also, the definition of a loaded server depends pretty much on the hardware. A Pentium3-500 on a dialup line can be probably called 'under load' with 10 mails per minute while I call a load 100 mails per second. However, all that could happen is that hMS builds up a queue and I think there is no limit except the available space.

It's never good to have unscanned email delivered! It's not good to have an email server on a system that is not cutting edge if the traffic is high. And especially it's not good to have anti spam and anti virus time out when the spammers send emails at three at a time designed to time out the checks and lets the spam with viruses and phishing and malware mail be delivered.
tBB wrote:
Therefore the processing timeout values of a mailserver should be IMO adaptable to the hardware/connection speed.

That would be a good feature to be added AFAIK. Or have a feature of a delivery queue for un scanned emails for rescan before final delivery in case of a time out.


I opt for that all mails most be scan without exception.

_________________
Any comment or statements is my own and have no relationship to my workplace


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-12 11:45 
Normal user

Joined: 2009-05-12 23:06
Posts: 52
Location: Denmark
tBB wrote:
horndog wrote:
Your right about this being slightly of topic but seeing as how you and Tooms probably get email notifications a new thread would not be desirable at this time.

I guess Tooms and me would quickly monitor the new thread as well ;)

However, as Martin stated that the feature will be integrated this poll is probably useless anyway. We could start a new one about configurable timeout values though :lol:

Best regards,
Nico


I try to keep up but you guys are fast here :?

_________________
Any comment or statements is my own and have no relationship to my workplace


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-12 11:53 
Normal user

Joined: 2009-05-12 23:06
Posts: 52
Location: Denmark
tBB wrote:
horndog wrote:
I just had a "Gregory House" moment! The biggest problem is ClamAV occasionally times out while reloading it's DB

Not sure what "Gregory House moment" means but let me quote myself:
tBB wrote:
However, it might happen that ClamD is reloading it's databases if there was a update at the time a connection comes in and during this time it doesn't scan anything (of course). Depending on the hardware it might also take way more than 30 sec. to load and unpack databases, specially if there are a lot of third party signatures involved so if 30 sec. is the hardcoded timeout I would really suggest to make that configurable.

So it's not ClamAV that times out, it's hMS!
horndog wrote:
So it appears that there are two main reasons for SpamAssassin to error all related to ClamAv.
Related to ClamAV? ClamAV has to load and unpack a huge database. If third party signatures are loaded which can be quite complex it takes even longer. Other virus scanners also don't scan anything during they reload their databases and queue the connections and other mailservers handle that as well.

Best regards,
Nico

From my understading it is one topic that the ClamAV dev. team is working one to make it load faster and when there comes new updates it will load the delta updates in the background and without stopping the scan low, so there is hope that this will get better in newer versions

_________________
Any comment or statements is my own and have no relationship to my workplace


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-12 12:12 
Senior user
Senior user
User avatar

Joined: 2009-04-17 18:10
Posts: 268
Location: The land of Beer and Sauerkraut!
Tooms wrote:
From my understading it is one topic that the ClamAV dev. team is working one to make it load faster and when there comes new updates it will load the delta updates in the background and without stopping the scan low, so there is hope that this will get better in newer versions

http://www.clamav.net/about/roadmap

Best regards,

Nico


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-12 22:26 
Normal user

Joined: 2009-05-12 23:06
Posts: 52
Location: Denmark
horndog wrote:
SpamD
Tooms wrote:
...But i understand that you like to scan incoming mails and reject the bad ones before they get in, I think can easly be don as i have seen other scripts on the forum where there is run a .exe and then use it exitcode to do the action.

I have a script VBScript to run an *.exe or *.bat file. I would need to know how MSWspamDscan would be used with a command line? It may or may not work properly but I'm willing to give it a try.



All the information about the commandline and exitcode can be found in the docs of mswclamdscan.

mswclamdscan.exe /mail:FullPath2MailFile.eml

Exitcode 0 = no bad things found
Exitcode 1 = Virus found
Exitcode 2 = Some error happen.

_________________
Any comment or statements is my own and have no relationship to my workplace


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-13 19:26 
Senior user
Senior user
User avatar

Joined: 2009-04-17 18:10
Posts: 268
Location: The land of Beer and Sauerkraut!
Accidentally I came across this thread viewtopic.php?f=10&t=16635 which is about AV scanner timeouts in hMS. Now I'm somewhat confused if there is a hardcoded 30 sec. timeout or not. Could someone who knows for sure maybe shed some light?

Best regards,

Nico


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-13 21:02 
Normal user

Joined: 2009-05-12 23:06
Posts: 52
Location: Denmark
horndog wrote:
Tooms wrote:
All the information about the commandline and exitcode can be found in the docs of mswclamdscan.

mswclamdscan.exe /mail:FullPath2MailFile.eml

Exitcode 0 = no bad things found
Exitcode 1 = Virus found
Exitcode 2 = Some error happen.

Thanks for that. But I need this for MSWSpamDScan. also how would I deal with the Exitcode in a command line?
I think I have all this "time out" problem figured out using rules and the script from:
viewtopic.php?p=101044#p101044
Basically I will write a rule the looks for the "X" header for clamAV and spamassassin. If the header does not exist then the email will be moved to a temp folder and then using the script will call ClamAV and/or spamassassin to rescan with no time out constraints. If the mail is clean them it will be moved to the inbox for that account. If the case is a virus or spam I can move it to a quarantine folder to be dealt with latter.


if you look at this post then you can see most if what your asking for, so this is a good start point.
viewtopic.php?f=14&t=14916

mswclamdscan will not modify the mail file in any way and this is by design to make the app very safe.

_________________
Any comment or statements is my own and have no relationship to my workplace


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-13 21:05 
Normal user

Joined: 2009-05-12 23:06
Posts: 52
Location: Denmark
horndog wrote:
horndog wrote:
Basically I will write a rule the looks for the "X" header for clamAV

I thought clamav produced an X header but it doesn't. Also there is a limitation with HMS rules that would limit any email moving after rescanning. This would have to be done with a script instead. The best option to overcome this problem is to have two ClamAV installs using mswclamdscan.exe.


if you use clamdscan there is maybe a option to let it add the header line, but mswclamdscan is design not to modify the mail.

Try look at the docs for clamdscan

mswclamdscan only return a exitcode and if the commandline option is used then it also can return a logfile with a single line for text.

_________________
Any comment or statements is my own and have no relationship to my workplace


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-13 21:07 
Normal user

Joined: 2009-05-12 23:06
Posts: 52
Location: Denmark
tBB wrote:
Accidentally I came across this thread viewtopic.php?f=10&t=16635 which is about AV scanner timeouts in hMS. Now I'm somewhat confused if there is a hardcoded 30 sec. timeout or not. Could someone who knows for sure maybe shed some light?

Best regards,

Nico


will be better if there was a option for the enduser to set the timeout value and also set what exitcodes there is errors and not only the detected exitcode as today

_________________
Any comment or statements is my own and have no relationship to my workplace


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-14 00:08 
Senior user
Senior user
User avatar

Joined: 2009-04-17 18:10
Posts: 268
Location: The land of Beer and Sauerkraut!
horndog wrote:
That was an old post. The HMS now have a 30.0 second time out. I can show you my logs that have 30.1 or more seconds then TCP/IP disconnect and SpamAssassin errors.

Ah I see, a new hMS version was released after that date. Could you perhaps give a rough estimate about how the available 30 seconds are divided between the processes (DNSBL lookups, ClamAV, SpamAssassin)? I have a nagging suspicion that SpamAssassin is the process which takes the longest time (at least if it's the PerlApp Win32 version from sourceforge) so it might be worth trying to speed it's execution up.

Best regards,

Nico


Last edited by tBB on 2010-01-14 00:22, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-14 00:21 
Senior user
Senior user
User avatar

Joined: 2009-04-17 18:10
Posts: 268
Location: The land of Beer and Sauerkraut!
horndog wrote:
I could design a batch file that could take advantage of the "error level." and then move or delete the file accordingly.
I must say that Nico writes some of the best batch files I have ever seen. Case in point is the one for ClamDog 10.

Such batchfile would be indeed not that hard. If I remember right I have already offered somewhere in this thread to provide some batch wrapper for ClamAV which parses the output/evaluates the errorlevel and adds a X-Header to the mail which contains the virus name. I assume supporting SpamC or Tooms programs would be not much different.

Best regards,

Nico


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-14 01:42 
Senior user
Senior user
User avatar

Joined: 2009-04-17 18:10
Posts: 268
Location: The land of Beer and Sauerkraut!
horndog wrote:
I was just looking through all the change logs but couldn't find any reference to TCP/IP time outs. This issue would be put to rest quickly if there was only a time stamp in the log for when ClamAv was reloading it's DB. That way I could reference that to the SpamAssassin errors.

There is a time stamp but you're probably looking in the wrong log (or it is disabled because this is the default). Look for the following lines in FreshClam.conf...

# Path to the log file (make sure it has proper permissions)
# Default: disabled
# UpdateLogFile c:\clamav\log\freshclam.log

# Log time with each message.
# Default: no
#LogTime yes

# Enable verbose logging.
# Default: no
#LogVerbose yes

..and enable what you see fit.

horndog wrote:
My SpamAssassin is the Windows port from:http://sawin32.sourceforge.net/
SpamAssassin-3.2.3.5-win32.zip

I thought it. This one is "compiled" by using PerlApp from Activestate. PerlApp wraps a whole 'Mini-Perl' package around the main Perl application which is unpacked to the system's temp path when the application is called (if compiled without some switches every time the application is called) and used from there. So, what you can do to speed SpamD up if you have the possibility is installing a ramdisk with a size of let's say +512 Mb and set the system's global temp path (NOT the pagefile!) to a temp folder on the ramdisk.

For such purposes I'm using the free 'Gavotte' ramdisk which is based on Microsoft's public ramdisk sources but has some nice additions. For example it can allocate the memory between 3.2Gb and 4Gb on 32 Bit systems for the ramdisk or it can create a image of the ramdisk with all paths and contents (e.g. the unpacked PerlApp SpamD) and reload it at every boot. You can get it here: http://www.megaleecher.net/RAMDisk

BTW: If you also set ClamAV's temp path (clamd.conf) to some folder on the ramdisk you will get another noticeable performance boost.

Best regards,

Nico


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-14 03:05 
Moderator
User avatar

Joined: 2007-06-14 05:12
Posts: 11891
Location: 'The Outback' Australia
horndog wrote:
I need a time stamp for this:
clamd.log

I get a timestamp in my clamd.log

The relevant config is clamd.config
Code:
Thu Jan 14 11:07:53 2010 -> Reading databases from c:\clamav\data
Thu Jan 14 11:07:53 2010 -> Not loading PUA signatures.
Thu Jan 14 11:08:00 2010 -> Loaded 698257 signatures.

_________________
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
Documentation


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-14 03:23 
Moderator
User avatar

Joined: 2007-06-14 05:12
Posts: 11891
Location: 'The Outback' Australia
You need to uncomment what you want by removing the leading #

Code:
##
## Example config file for the Clam AV daemon
## Please read the clamd.conf(5) manual before editing this file.
##

# Uncomment this option to enable logging.
# LogFile must be writable for the user running daemon.
# A full path is required.
# Default: disabled
LogFile c:\clamav\log\clamd.log

# By default the log file is locked for writing - the lock protects against
# running clamd multiple times (if want to run another clamd, please
# copy the configuration file, change the LogFile variable, and run
# the daemon with --config-file option).
# This option disables log file locking.
# Default: no
#LogFileUnlock yes

# Maximum size of the log file.
# Value of 0 disables the limit.
# You may use 'M' or 'm' for megabytes (1M = 1m = 1048576 bytes)
# and 'K' or 'k' for kilobytes (1K = 1k = 1024 bytes). To specify the size
# in bytes just don't use modifiers.
# Default: 1M
LogFileMaxSize 4M

# Log time with each message.
# Default: no
LogTime yes

# Also log clean files. Useful in debugging but drastically increases the
# log size.
# Default: no
LogClean yes

# Use system logger (can work together with LogFile).
# Default: no
#LogSyslog yes

# Specify the type of syslog messages - please refer to 'man syslog'
# for facility names.
# Default: LOG_LOCAL6
#LogFacility LOG_MAIL

# Enable verbose logging.
# Default: no
LogVerbose yes

_________________
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
Documentation


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-14 03:34 
Moderator
User avatar

Joined: 2007-06-14 05:12
Posts: 11891
Location: 'The Outback' Australia
Been watching this thread... :mrgreen:

Just noticed that my clamd service wasn't running for a month. (it is my secondary AV anyway)
I haven't notice any time out issues in the hmailserver logs. What should I look for?

_________________
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
Documentation


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-14 11:14 
Senior user
Senior user
User avatar

Joined: 2009-04-17 18:10
Posts: 268
Location: The land of Beer and Sauerkraut!
FreshClam.log shows the time stamp as well:
horndog wrote:
Wed Jan 13 15:32:16 2010 -> daily.cld updated (version: 10296, sigs: 153941, f-level: 44, builder: edwin)
Wed Jan 13 15:32:16 2010 -> Database updated (698976 signatures) from database.clamav.net (IP: 194.186.47.19)
Wed Jan 13 15:32:16 2010 -> Clamd successfully notified about the update.

Wed Jan 13 17:32:00 2010 -> --------------------------------------

Best regards,

Nico


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-14 11:36 
Senior user
Senior user
User avatar

Joined: 2009-04-17 18:10
Posts: 268
Location: The land of Beer and Sauerkraut!
mattg wrote:
I haven't notice any time out issues in the hmailserver logs.

:shock:

Best regards,

Nico


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-14 12:06 
Normal user

Joined: 2009-05-12 23:06
Posts: 52
Location: Denmark
horndog wrote:
I've also noticed that it takes SA over 20 seconds to run by it's self. I'll have more observations soon.


wow that was slow, i have a VM with a linux and SA and when i send a mail to it via a SpamC client then it scan in 1 sec and some times 2 sec.

I think you better try to debug your SA setup and see if it hangs in some old RBL lists or some thing like that as i have before seen SA wait and timeout on RBL or other things and that giv that long scan time and when outcoment the bad checks then SA was much faster.

_________________
Any comment or statements is my own and have no relationship to my workplace


Top
 Profile  
 
 Post subject: Re: Interface to ClamD just like the one for SpamD
PostPosted: 2010-01-14 12:27 
Senior user
Senior user
User avatar

Joined: 2009-04-17 18:10
Posts: 268
Location: The land of Beer and Sauerkraut!
horndog wrote:
The first thing I noticed was the total lack of data indicating that SA would error when ClamAv was reloading it's DB. I've also noticed that it takes SA over 20 seconds to run by it's self.
That's exactly what I suspected. I was porting SA myself using PerlApp since version 2.something to 3.something for a company until I gave up because usually a week after a version was ported and implemented, the SA guys came up with a new minor version.

The Linux version is that much faster because AFAIK there's a resident Perl daemon running. Even when using the mentioned ramdisk trick the Win32 SpamD won't come even close to the processing time of the Linux version.

If you are using third party rules like for example the SARE rules from http://www.rulesemporium.com this can also slow down the processing a lot.

Last but not least, as Tooms mentioned you should check if you're using a unresponsive RBL.

Best regards,

Nico

EDIT: As far as I remember the SA port you're using also has a couple of plugins integrated. Some of them are slowing down the process a lot, e.g. the OCR plugin (which is meanwhile almost useless anyway).


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 73 posts ]  Go to page 1, 2  Next


Who is online

Users browsing this forum: No registered users and 0 guests



Search for:
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group