ASSP Attachment Problems

Forum for things that doesn't really have anything to do with hMailServer. Such as php.ini, beer, etc etc.
Post Reply
User avatar
Jason Weir
Normal user
Normal user
Posts: 58
Joined: 2004-02-02 23:41
Location: Chichester, NH
Contact:

ASSP Attachment Problems

Post by Jason Weir » 2004-06-25 23:45

Polarunion has mentioned problems with ASSP and attachments. Is anyone else having this problem. I cannot duplicate the problem. The problem might lie within the version of perl that is installed. There are versions that have known memory leaks. If anyone is having this problem can you post your config. Most importantly I'd like to know

OS
CPU
RAM
ASSP version
Perl Type and Version (ie. Activestate 5.8.1)
Web Server Type and Version
hMailServer Version

Thanks
Jason Weir

calvi
Normal user
Normal user
Posts: 65
Joined: 2004-03-17 23:34
Location: Melbourne, Australia

Post by calvi » 2004-06-26 02:30

Yes I think polarunion is a bit harsh on ASSP, but he does have a point, the inclusion of virus filtering has added alot more strain on the system.

I have been running ASSP for nearly a year. Believe it or not it runs on the following config....

Pentium 150 (not even mmx!)
256m/b ram.
6g/b SCSI HD.
Windows NT4 SP6a
ASSP version 1.0.10d.1 (The .1 is because I have customised the code).
Activestate Perl 5.635 (cannot run 5.8 with Windaemon).

This machine also runs...
Zonealarm Pro Firewall.
Apache 1.3.28 with SSL, PHP 4.3
Eudora WorldMail Mailserver.
Some port forwarding services.

ASSP without virus checking does slow down on large attachments a bit on this config, but thats hardly surprising, its really a testament to ASSP that it can run OK on that setup.

Turning on virus checking in ASSP 1.0.10d however really killed my poor old machine. It started up and then after a day slowed to a gasping crawl.

So you do need a bit of grunt to run ASSP-ClamAV enabled. How much grunt? Well here are some tests.....

I am in the process if moving everything to our server (and off the old dial up email machine) but still haven't finished getting everything accross to it (too much real work to do).

The server is a Opteron 242 - 1 g/b ram - RAID5 SATA etc....
Running....
Windows Server 2003
Apache 2.0.49, SSL, PHP 4.3.6.
hMailserver (of course!)
ASSP 1.0.10d.1
Activestate Perl 5.635.
MySQL 4.0.18

Now using an outlook express client running locally on this machine and sending a 10mb attachment I get the following results...

1. Sending through ASSP(no Clam)>Worldmail running on ol' pentium = 17 minutes 25s. (don't even bother with clam on, she can't take it captain).
2. Sending direct to Worldmail running on ol' pentium = 40s
3. Receiving from worldmail running on ol' pentium = 25s.
4. Sending through ASSP(no Clam)>hMailserver on Sledgehammer = 21s
5. Sending through ASSP(Clam)>hMailserver on Sledgehammer = 2min 44s.
6. Sending direct to hMailserver on Sledgehammer = 10s.
7. Recieving from hMailserver on Sledgehammer = 3s.

Now bear in mind that all the send times include encoding in outlook and moving the email their respective folders, this was consistent at around 6-7s.

This means for me ASSP is great but with Clam AV a bit heavy on the CPU side.

And my apologies to Martin as there are no speed issues with hMailserver at all.

User avatar
Jason Weir
Normal user
Normal user
Posts: 58
Joined: 2004-02-02 23:41
Location: Chichester, NH
Contact:

Post by Jason Weir » 2004-06-26 04:56

Calvi,

FYI I'm running Activestate 5.8.0 along with win32::daemon no problems

I'm running ASSP v1.0.9 so I don't have ClamAV support, thats why I don't see any speed issues. Do we think the problem lies with ASSP or ClamAV? Has anyone tested speeds when ClamAV is used through hMailserver? Maybe the problem is ClamAV and not ASSP or hMailserver.

BTW Only in the Open Source Community do you see people still using old hardware. Try running Exchange 2000 on a Pentium 150....

Jason

calvi
Normal user
Normal user
Posts: 65
Joined: 2004-03-17 23:34
Location: Melbourne, Australia

Post by calvi » 2004-06-26 05:21

Jason,

I had some problem getting win32:daemon running with 5.8 a long time ago. It might have been because it was windows NT or something. Anyway I will have another go.

I'm running ASSP v1.0.9 so I don't have ClamAV support, thats why I don't see any speed issues. Do we think the problem lies with ASSP or ClamAV
Definately an ASSP issue. ASSP does not actually use ClamAV. It checks using perl, the message against the ClamAV databases. (Fairly clever I think).

I have been using ASSP for nearly a year and I LOVE it, don't get me wrong. I too have no problems with AV turned off.

I have looked at the code and there are some shortcomings to the AV implementation. I have made a couple of posts on the ASSP forum to this effect...

http://sourceforge.net/forum/forum.php? ... _id=235332

and a response to polarunions post....

http://sourceforge.net/forum/forum.php? ... _id=235332

Now I will try PERL 5.8 and maybe it will help?

The good news is there is no performance issue with hMailserver, and no compatibility issues between ASSP and hMailserver.

With ASSP running on my Opteron server the time to do a 10m/b attachment is 164s. This is not the end of the world as it is a throughput of about 60kb/s which is ok. The problem is local users should not experience the delay and the server CPU gets crunched for a long time.

I will be upgrading to a dual opteron in the not too distant future but would prefer not to have my resources taxed this highly.

Jason please try 1.0.12 with ClamAV and let us know if it causes you any speed issues?

John C.

User avatar
Jason Weir
Normal user
Normal user
Posts: 58
Joined: 2004-02-02 23:41
Location: Chichester, NH
Contact:

Post by Jason Weir » 2004-06-26 05:41

I was going to update to 1.0.10 until bugs were found, then 11 came out and now 12. I think I will wait a couple of weeks and see what else is found. I've been helping John Hanna with some of the coding for ASSP, so some of the bugs might be my fault :)

I can try ASSP with ClamAV but if I find speed issues I will probably upgrade to hMailserver 3.1.x or 3.2 and use ClamAV from within hMailserver.

My config is as follows:

2.4ghz P4 w\512MB RAM
Windows 2K Pro
IDE 80GB drive
Activestate Perl v5.8.0
hMailserver v2.0.5
ASSP v1.0.9

I know I'm a couple versions back on both ASSP and hMailserver but I like stable and tested versions. Martin is releasing new versions on a monthly basis, I'd like to see a couple months go by after a release candidate is released and real world tested. If no bugs are found after a couple of months then I'd consider it stable.

Off my soap box for now.

Jason

polarunion
Normal user
Normal user
Posts: 245
Joined: 2004-04-05 20:21
Location: Ottawa, Canada
Contact:

Post by polarunion » 2004-06-26 07:15

thanks Jason and John..

John I really appreciated your response and list of suggestions for ASSP on the SF forum. I think each of your suggestions would improve ASSP considerably. I especially liked this suggestion
"ClamAV performance - Clam AV performance is quite terrible on my server. I think that a store and forward approach may need to be looked at to prevent my users complaining about the time it takes to send a message to the mailserver. That is if the email passes all ASSP's other checks it stores the email, checks it for virus then passes it on. This has some shortcomings and is quite tricky to implement so in the short term I am looking at bypassing AV for local users. This would be a good option also in the admin interface."
I completely agree. It just makes much more sense on the server side, and on the client side. It would be wonderful not to have to wait while perl/clamav checks the message as it's coming in. Let the server do that on its own time and let the user just get on with their day instead of waiting and waiting..

However, nothing explains why i'm experiencing upload times of 12 minutes when using outlook vs. mozilla's thunderbird - only taking about a minute - or even SM. Maybe its not an hMail issue, and more an outlook issue, but if most of my users are using outlook, I'd like to get to the bottom of this.

Ps. John - can you try your tests again remotely? Is this how you have been doing your tests as well Martin? I experienced no loss of speed when sending locally. The issue only occurs when a person tries to send (remotely) a large email to the local server.

Also - in checking the server resource stats tonight, hMailServer was running at 143 MB of ram.. What is accumulating in hmail to cause this? that can't be good. I barely even have enough users to really put a strain on it... This occurs after a days use - like todays testing with heavy attachments. After restarting the service, it was back down to 4.5.

(Just an aside - it also did not remember which emails were marked as read and unread. ie - messages that appeared as new after the last restart - and which were read, were once again marked as unread.)

Does anyone have any ideas on what I should try?
John, Jason, I may try your configurations...Is it activestate? Is there any other? I have activestate 5.8 with the windaemon installed. I've noticed issues with using 5.8 on windows so this may have something to do with assp/clamav churning the machine to pieces. I did notice perl working at 100% when messages were sent.

Lastly, I may try going back a couple of versions because I don't remember outlook ever taking this long to upload a large attachment. In fact I remember trying a 10mb attachment on a 3.0 release and being amazed at how fast it was processed.

Again, thanks again team for shedding some more light on this and john, for your efforts in posting your suggestions on ASSP in the SF forum. I really hope this does not get burried and ignored. I too agree that ASSP is otherwise the best spam blocker available. I've had great success with it.

After all this however, I think there will be a real need eventually to include a Max Attachment size in hmailserver. Seems pretty easy to bring my server down with a couple of large attachments.

Shhh don't tell anyone this though!

Off to the cottage for a few days. Go England. (they're out? already?)

Pat.

calvi
Normal user
Normal user
Posts: 65
Joined: 2004-03-17 23:34
Location: Melbourne, Australia

Post by calvi » 2004-06-26 16:16

Jason/Pat,

I think I have fixed the problem with ASSP clamAV.

ref. http://sourceforge.net/forum/forum.php? ... _id=235332

Its a simple change.

calvi
Normal user
Normal user
Posts: 65
Joined: 2004-03-17 23:34
Location: Melbourne, Australia

Post by calvi » 2004-06-27 01:26

Polarunion
Ps. John - can you try your tests again remotely? Is this how you have been doing your tests as well Martin? I experienced no loss of speed when sending locally. The issue only occurs when a person tries to send (remotely) a large email to the local server.
Can you define remotely?
Does anyone have any ideas on what I should try?
John, Jason, I may try your configurations...Is it activestate? Is there any other? I have activestate 5.8 with the windaemon installed. I've noticed issues with using 5.8 on windows so this may have something to do with assp/clamav churning the machine to pieces. I did notice perl working at 100% when messages were sent.
Yes it is activestate, I have tried 5.6 & 5.8 with no noticeable difference.
Can I ask what issues you have had with 5.8 as distinct from 5.6 as I am yet to upgrade my main server?

polarunion
Normal user
Normal user
Posts: 245
Joined: 2004-04-05 20:21
Location: Ottawa, Canada
Contact:

Post by polarunion » 2004-06-28 18:49

hey John, how is your upgrade working out for you? nice work!!! I'm anxious to get ASSP back on there asap. I really took it for granted. I opened up my mail this morning and sure enough over 30 messages from the last few days.

Anway I was going to suggest that you try sending a message to your server from a computer outside of your network - ensuring that it goes through the net all the way. T'is what I meant by remotely. Sorry for the confusion.

Only reason I suggested that is because I too tried to send a message while I was on the server and I experienced extremely fast times. I only ran into lag problems when I was using another computer on the network. Why that made a difference, I couldn't tell ya, but it certainly did.

Thanks again for all that effor you put into that project. Great thing about open source - you're usually helping out many others while helping yourself.

calvi
Normal user
Normal user
Posts: 65
Joined: 2004-03-17 23:34
Location: Melbourne, Australia

Post by calvi » 2004-06-29 00:30

Upgrade is working out well,

I have been migrating to open source for a few years now.

Its all running on Server 2003. One day I'll bite the bullet and go Linux.

The reason I ask define remote is because theoretically all connections are tcp/ip regardless of where they are. Of course we know everthing is different in practise, especially when it comes to computers.

Are you sure your "local test" was through ASSP and not direct to the mailserver. Do you have some virus software or something on the client you tried?

I will try a remote connection anyway to make sure ASSP and hMailServer transfer speeds are ok anyway.

PS. I upgraded to activestate 5.8 and have found the performance about the same as 5.6.

JC.

polarunion
Normal user
Normal user
Posts: 245
Joined: 2004-04-05 20:21
Location: Ottawa, Canada
Contact:

Post by polarunion » 2004-06-29 00:51

thanks john, i'll be doing a fresh format of the current system and start from scratch. Thanks for the tips. I'll try out the new version.

ps. where would I integrate your code in the assp code? I don't know perl so I'm affraid I might be a bit useless but am axious to try whatever is working for you.

calvi
Normal user
Normal user
Posts: 65
Joined: 2004-03-17 23:34
Location: Melbourne, Australia

Post by calvi » 2004-06-29 01:15

I've emailed you my version.

Of course as they say, "Use at your own peril" :)

John C.

User avatar
Jason Weir
Normal user
Normal user
Posts: 58
Joined: 2004-02-02 23:41
Location: Chichester, NH
Contact:

Post by Jason Weir » 2004-06-29 01:19

John,

Did you ever get a response from John Hanna regarding the AV sub being called twice? I just wondered if it was by design for some reason or just a bug?

Jason

calvi
Normal user
Normal user
Posts: 65
Joined: 2004-03-17 23:34
Location: Melbourne, Australia

Post by calvi » 2004-06-29 01:32

No not yet.

polarunion
Normal user
Normal user
Posts: 245
Joined: 2004-04-05 20:21
Location: Ottawa, Canada
Contact:

Post by polarunion » 2004-06-30 20:56

Jason, you should really try to get in touch with the developper of ASSP so he can pay some attention to this. He doesn't seem to active in his forum like Martin spoils us over here.

I just tried out John's modded assp.pl and i've noticed huge performance increases.

See here for the times of the latest tests.

http://www.hmailserver.com/forum/viewto ... c&start=15

Obviously I'm sure more can be done for tweaking but this is a great start.
Thanks again JC.

Post Reply