Spamassassin Memory Leak

Forum for things that doesn't really have anything to do with hMailServer. Such as php.ini, beer, etc etc.
Post Reply
roi
Normal user
Normal user
Posts: 153
Joined: 2009-09-20 12:56
Location: Chiba, Japan

Spamassassin Memory Leak

Post by roi » 2009-10-05 09:30

Okay, this is a spamassassin issue so perhaps I shouldn't even talk about this here. But then again, I figure many hMailServer installs run SA because support for SA is integrated, so I can compare notes among my fellow hMailserver users who are also using SA. To the forum moderators, if you feel this post is out of line, please feel free to zap it.

I read in various places in the web that spamd has a slow, but sure memory leak and this seems to have been reported more than a year ago. Unfortunately, our forum here does not seem to have any post on this issue. So I decided to watch the memory usage of my spamd.exe, and indeed, the memory usage of spamd.exe seems to be going up very slowly, but very surely.

-- spamd.exe version: 3.2.3.5 (file version), 3.2.3.0 (Product version)
-- start-up usage: 101Mb (saw it go up to 104, 106, 111, 112, to now 113) in 24 hours after startup of hMS service.
-- mail volume: very small ( <500 in 24 hours )
-- spam to ham ratio: 90:10 (I'm hosting a very old domain that onced lived in sendmail and postfix in the span of 13 years, so most spam are aimed at that domain.)
-- CPU usage hovering at 2%-7%, peaks to 100% for a few seconds on processing spam/ham.
-- Page File Usage: was 257 MB at service start up, now steady at 291 MB (PF also seems to be slowly going up as it is now 294 MB during the time I was drafting this post.)
-- Total Physical Memory usage: about 30%

The newest version of spamd for win32 seems to be 3.2.3.5, but other scattered web posts seem to suggest that this memory leak has been fixed with spamd version 3.2.5 (but I can't seem to find a win32 port of this version.)

So how do most of my fellow hMS community members find spamd.exe as far as possible slow memory leak is concerned?

How much memory did your spamd.exe use when you last rebooted the OS or start spamd process?

How much memory does your spamd.exe use now?

So far, I have not had any serious problem with hMailServer with spamd.exe, but this could mean I can't leave my eyes off spamd.exe and may need to stop/start it once in a while. For those with limited physical memory, this can be a serious issue.

My system is described in my signature below.
hMS: 5.2.1-B361 | DB: Internal MySQL from hMS 4.4 | OS: W2K3 1Gb VM

roi
Normal user
Normal user
Posts: 153
Joined: 2009-09-20 12:56
Location: Chiba, Japan

Re: Spamassassin Memory Leak

Post by roi » 2009-10-05 16:18

spamd is now using 117 Mb since my last post. It is slowly but surely rising.
hMS: 5.2.1-B361 | DB: Internal MySQL from hMS 4.4 | OS: W2K3 1Gb VM

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: Spamassassin Memory Leak

Post by ^DooM^ » 2009-10-05 16:41

How do you know SA / Windows wont do a garbage run when it reaches a certain point?
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

roi
Normal user
Normal user
Posts: 153
Joined: 2009-09-20 12:56
Location: Chiba, Japan

Re: Spamassassin Memory Leak

Post by roi » 2009-10-05 18:12

Don't really know at this stage. This is why I am watching this carefully after reading articles (not in our forum here) that spamd has a slowly accummulating memory leak. This is also why I was wondering what others are seeing/experiencing. I'm hoping what you say is true because I kind of like spamassassin as a generic spam filter of last resort and if the leak has been fixed but the new version has no win32 release, I don't know what to do until the fixed version is released.

Thanks for the question/interest...
hMS: 5.2.1-B361 | DB: Internal MySQL from hMS 4.4 | OS: W2K3 1Gb VM

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: Spamassassin Memory Leak

Post by ^DooM^ » 2009-10-05 18:14

I used SA32 but found it unstable. I'm waiting for a maintenance window on my nix box that's coming up soon to install it on there. Much more reliable.
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

User avatar
sheffters
Senior user
Senior user
Posts: 453
Joined: 2009-07-01 20:46
Contact:

Re: Spamassassin Memory Leak

Post by sheffters » 2009-10-05 18:15

Can you not build / compile a win32 from source of latest version? afik its all cross platform so should just comile in C. The free express edition will probably work for compiling.

dont know for sure, but worth a shot?

S.

roi
Normal user
Normal user
Posts: 153
Joined: 2009-09-20 12:56
Location: Chiba, Japan

Re: Spamassassin Memory Leak

Post by roi » 2009-10-05 18:40

Compiling from the patched source is definitely an option, but an option with a steep learning curve for me. I'll look into that possibility anyway.

spamd is now using 118 Mb ...

Thanks...
hMS: 5.2.1-B361 | DB: Internal MySQL from hMS 4.4 | OS: W2K3 1Gb VM

User avatar
martin
Developer
Developer
Posts: 6837
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: Spamassassin Memory Leak

Post by martin » 2009-10-05 18:53

I run a VMware instance with Ubuntu / SpamAssassin for a while. Even if that SpamAssassin was running under a virtual operating system, SA under Linux was 10 times faster than the Windows port, and caused a lot less problems. I'm not sure about the memory usage, but I wouldn't be surprised if the memory usage with running Linux + Linux-SA is lower than running SA on Windows. :)

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: Spamassassin Memory Leak

Post by ^DooM^ » 2009-10-05 19:01

I'll tell ya when i get it up and running ;)
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

Tooms
Normal user
Normal user
Posts: 52
Joined: 2009-05-12 23:06
Location: Denmark
Contact:

Re: Spamassassin Memory Leak

Post by Tooms » 2009-10-06 14:01

How much memory SpamD is using is depending alot on what commandline options you have started it with and how many child-threads it has running.

my spamd.exe on high-load mailscanner is using 380mb


to control your spamd.exe then try look at my app there is design to help with this
http://www.tooms.dk/software/spamdloade ... efault.asp

I made the app when i was writing this guide and having many of the same issues
http://www.tooms.dk/articles_tutorials/ ... efault.asp


Hope it can help you make it more stabile.
Any comment or statements is my own and have no relationship to my workplace

roi
Normal user
Normal user
Posts: 153
Joined: 2009-09-20 12:56
Location: Chiba, Japan

Re: Spamassassin Memory Leak

Post by roi » 2009-10-06 14:14

Thanks tooms... actually, way before your reply to my post, I already visited your web site and downloaded your gizmo... utility or whatever one might call it because DooM referenced it in one of his earlier post (not related to my post) discussing spamd.

Because I have not yet used it, I did not refer to it yet in any of my posts, but rest assured your work has been recognized even before you came to offer it. So now, I have the chance to thank you (and DooM) here now.

I still have a long way to go before my spamd gets to use the ram your spamd is using... I can sigh a sigh of relief?

Thank you tooms, DooM...
hMS: 5.2.1-B361 | DB: Internal MySQL from hMS 4.4 | OS: W2K3 1Gb VM

Tooms
Normal user
Normal user
Posts: 52
Joined: 2009-05-12 23:06
Location: Denmark
Contact:

Re: Spamassassin Memory Leak

Post by Tooms » 2009-10-06 14:55

okey and thanks for the nice word, just glad to help :-)
Any comment or statements is my own and have no relationship to my workplace

roi
Normal user
Normal user
Posts: 153
Joined: 2009-09-20 12:56
Location: Chiba, Japan

Re: Spamassassin Memory Leak

Post by roi » 2009-10-09 05:20

DooM wrote: How do you know SA / Windows wont do a garbage run when it reaches a certain point?
Well, DooM was right on after all. Yesterday, spamd was up to 120 Mb, but suddenly today, it went down to 114 Mb, so SA/Windows did garbage collection. It took a few days (5 days to be exact) to get to do garbage collection; I guess memory leak should not be a concern with spamd.exe (this version I am using anyway).

Just thought I should update those who were kind enough to comment on my status. Thanks.
hMS: 5.2.1-B361 | DB: Internal MySQL from hMS 4.4 | OS: W2K3 1Gb VM

Tooms
Normal user
Normal user
Posts: 52
Joined: 2009-05-12 23:06
Location: Denmark
Contact:

Re: Spamassassin Memory Leak

Post by Tooms » 2009-10-10 11:11

horndog wrote:
I just installed your spamdloaderservice. Very nice program! No problems at all.

Spamd is using 293MB of memory. using the default command line parameters.

Thanks again.
The default commandline options is for "high load" mail scanner so it is maybe a good idea to fine tune them to better fit your needs and so it only use the memory on your system there is need.
Any comment or statements is my own and have no relationship to my workplace

Tooms
Normal user
Normal user
Posts: 52
Joined: 2009-05-12 23:06
Location: Denmark
Contact:

Re: Spamassassin Memory Leak

Post by Tooms » 2009-10-11 17:50

horndog wrote:
Tooms wrote: The default commandline options is for "high load" mail scanner so it is maybe a good idea to fine tune them to better fit your needs and so it only use the memory on your system there is need.
Any suggestions? I don't have a high load Mail server.
not sure about this settings so you have to monitor it for error.

But try change 5 in the "--min-children=5" to be 1 or 2

and in the gui set "Minimum thread the SpamD can have before the process is recycled (0=disable)" to 1 or 0


try look at this
http://spamassassin.apache.org/full/3.1 ... spamd.html
Any comment or statements is my own and have no relationship to my workplace

Post Reply