PLease help me reduce the spam - I'm not getting very far

Use this forum if you have installed hMailServer and want to ask a question related to a production release of hMailServer. Before posting, please read the troubleshooting guide. A large part of all reported issues are already described in detail here.
Post Reply
AlanSilver
Normal user
Normal user
Posts: 190
Joined: 2007-04-22 17:06

PLease help me reduce the spam - I'm not getting very far

Post by AlanSilver » 2008-04-29 22:15

Hello,

I've been battling with spam for a while and am not getting very far. I'm hoping someone can help me work out how to reduce (eliminate?) spam, without losing real e-mails.

I have added loads of addresses to the white list, and this seems to be working fine.

I tried enabling grey listing, but this caused every message that wasn't from an address in my white list to be given the "Please try later" response. Most of these never got through. I seemed to be losing a lot of mail.

Without grey listing enabled, my e-mails come through OK, but so does a ton of spam.

Please can anyone explain (in simple terms, I'm no expert at this) how I should best configure it to cut out as much spam as possible, without losing real e-mails.

Also, are there any good add-ons that will do content-based spam filtering? As I understand it, hMailserver does not do content filtering at all.

Thanks,
Alan

plobby
Normal user
Normal user
Posts: 115
Joined: 2008-01-29 07:04

Re: PLease help me reduce the spam - I'm not getting very far

Post by plobby » 2008-04-29 23:23

Look into ASSP or Spam Assassin.

minsik
Normal user
Normal user
Posts: 92
Joined: 2006-08-03 13:08
Location: South Australia

Re: PLease help me reduce the spam - I'm not getting very far

Post by minsik » 2008-04-30 01:36

from experience, my company uses crap email software (not hmail) and we we buckling at the knees from the weight of spam.

I put in assp (not easy to implement but follow the basic guides and learn learn) After a breif test few days or 1 week we went live, since then we are sooooo happy. 50-70 email users 1 domain. We cc all spam to an email account, review it as needed (in the early days) now we hardly look at any spam. We delete it after a peeriod just in case there is any missclassification, its there as a backup only.

So follow the rules, set it up according to the guides. let it learn for a while collecting good and bad. you review the good and bad and correctly classify the emails. training the dog! good dog, bad dog! etc. adding rules, list sof users etc.

There is an excelllent forum and help but like here, rtfm. rtff rtfe (read the flaming manual, read the flaming forums, read the flaming everything) and you will get it working for you.

Our email software is so bad if we remove assp then its like a honey pot and in 5 days would be unworkable again.

It all runs on a 1.4gig 512meg pc windows xp.
Windows XP
hmailserver (latest beta allways) (except db versions)
test config, trying to convince the boss to use hmail!
ASSP - front end SPam killer

User avatar
mattg
Moderator
Moderator
Posts: 20837
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: PLease help me reduce the spam - I'm not getting very far

Post by mattg » 2008-04-30 02:32

AlanSilver wrote:I tried enabling grey listing, but this caused every message that wasn't from an address in my white list to be given the "Please try later" response. Most of these never got through. I seemed to be losing a lot of mail.
I've been using greylisting for over a year.
Don't know of ANY message that I didn't get.
Don't know of ANY SPAM that wasn't trapped and tagged using the built in Spam checking - DNS blacklists, SUPRBL etc
There are rare messages that are tagged as SPAM but aren't - but normally because of a dodgy domain name.

I only have 1 domain and about 50 users. Zero SPAM in twelve months (except through an external account).

Are you sure that you are losing mail?
Other than computer generated logs or reports etc, I don't know any mailservers that won't try again automatically.
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

gotzboost
Normal user
Normal user
Posts: 49
Joined: 2008-04-18 19:28

Re: PLease help me reduce the spam - I'm not getting very far

Post by gotzboost » 2008-04-30 03:05

What BL and SUPRBL are you using?

Chris1
Normal user
Normal user
Posts: 34
Joined: 2008-02-08 05:21

Re: PLease help me reduce the spam - I'm not getting very far

Post by Chris1 » 2008-04-30 03:19

"Are you sure that you are losing mail?
Other than computer generated logs or reports etc, I don't know any mailservers that won't try again automatically."

The statement above may technically be correct however do not forget that Yahoo and Hotmail resend the message from a different mail server so gray listing will not work.

I hope this helps.

gotzboost
Normal user
Normal user
Posts: 49
Joined: 2008-04-18 19:28

Re: PLease help me reduce the spam - I'm not getting very far

Post by gotzboost » 2008-04-30 04:16

We receive hotmail and yahoo emails all day long, and we have grey listing enabled.

User avatar
mattg
Moderator
Moderator
Posts: 20837
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: PLease help me reduce the spam - I'm not getting very far

Post by mattg » 2008-04-30 04:24

gotzboost wrote:What BL and SUPRBL are you using?
hmailserver_admin.jpg
hmailserver_admin.jpg (19.9 KiB) Viewed 8037 times
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

AlanSilver
Normal user
Normal user
Posts: 190
Joined: 2007-04-22 17:06

Re: PLease help me reduce the spam - I'm not getting very far

Post by AlanSilver » 2008-04-30 09:50

mattg wrote:
AlanSilver wrote:I tried enabling grey listing, but this caused every message that wasn't from an address in my white list to be given the "Please try later" response. Most of these never got through. I seemed to be losing a lot of mail.
I've been using greylisting for over a year.
Don't know of ANY message that I didn't get.
Don't know of ANY SPAM that wasn't trapped and tagged using the built in Spam checking - DNS blacklists, SUPRBL etc
There are rare messages that are tagged as SPAM but aren't - but normally because of a dodgy domain name.

I only have 1 domain and about 50 users. Zero SPAM in twelve months (except through an external account).

Are you sure that you are losing mail?
Other than computer generated logs or reports etc, I don't know any mailservers that won't try again automatically.
I don't know for a fact that I'm losing mail because (by definition), I don't get the ones that I suspect are lost :D. It's more of a feeling based on what I expect to arrive and what doesn't.

Maybe I'll persevere and see if I can find any hard evidence. I'm not sure I want to cope without grey listing. I turned it off last night, and this monring I had a ton of spam :(

Thanks
Alan

AlanSilver
Normal user
Normal user
Posts: 190
Joined: 2007-04-22 17:06

Re: PLease help me reduce the spam - I'm not getting very far

Post by AlanSilver » 2008-04-30 09:51

Chris1 wrote:"Are you sure that you are losing mail?
Other than computer generated logs or reports etc, I don't know any mailservers that won't try again automatically."

The statement above may technically be correct however do not forget that Yahoo and Hotmail resend the message from a different mail server so gray listing will not work.

I hope this helps.
I noticed this. I there any way around it with grey listing?

Chris1
Normal user
Normal user
Posts: 34
Joined: 2008-02-08 05:21

Re: PLease help me reduce the spam - I'm not getting very far

Post by Chris1 » 2008-04-30 10:14

Hello.

The only thing you can do is white list the domains that do not work with gray listing. If you search the forums you will find a comprehensive list of major ISPs that do not work with gray listing.

I hope this helps.

User avatar
SorenR
Senior user
Senior user
Posts: 3575
Joined: 2006-08-21 15:38
Location: Denmark

Re: PLease help me reduce the spam - I'm not getting very far

Post by SorenR » 2008-04-30 14:07

AlanSilver wrote:I tried enabling grey listing, but this caused every message that wasn't from an address in my white list to be given the "Please try later" response. Most of these never got through. I seemed to be losing a lot of mail.
The SMTP log will tell you. Elsewhere in this forum there is information about using Microsoft Log Parser to extract information from the log. Time to brush up your SQL skills :lol:

http://www.hmailserver.com/forum/viewto ... =7&t=11792
SørenR.

“Those who don't know history are doomed to repeat it.”
― Edmund Burke

phil54
Normal user
Normal user
Posts: 195
Joined: 2007-11-26 13:13
Location: UK :-)

Re: PLease help me reduce the spam - I'm not getting very far

Post by phil54 » 2008-05-02 12:58

AlanSilver, what dns blacklists etc are you using?
Image

AlanSilver
Normal user
Normal user
Posts: 190
Joined: 2007-04-22 17:06

Re: PLease help me reduce the spam - I'm not getting very far

Post by AlanSilver » 2008-05-04 18:16

phil54 wrote:AlanSilver, what dns blacklists etc are you using?
DNS Blacklist...
zen.spamhaus.org
bl.smapcop.net

SURBL...
multi.surbl.org

I've actually noticed a drop in spam since I turned grey listing back on, but haven't checked the logs to check that all genuine mail got through.

I downloaded log parser, but haven't had time to play with it yet.

Thanks
Alan

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: PLease help me reduce the spam - I'm not getting very far

Post by ^DooM^ » 2008-05-04 20:27

You should install this

http://www.hmailserver.com/forum/viewto ... =14&t=6698

It will help with determining if mail is delayed or not.
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

AlanSilver
Normal user
Normal user
Posts: 190
Joined: 2007-04-22 17:06

Re: PLease help me reduce the spam - I'm not getting very far

Post by AlanSilver » 2008-05-04 23:49

^DooM^ wrote:You should install this

http://www.hmailserver.com/forum/viewto ... =14&t=6698

It will help with determining if mail is delayed or not.
Thanks, but I'm running hMailserver with SQL Server, not MySql, so the script doesn't work (we discussed this in that thread, but I guess you've written a lot of replies since then, so I'll forgive you for not remembering!).

I re-read the thread, and refreshed my memory about how grey listing works, but I'm not sure how I would check to see what e-mails failed. If someone tried to send mail, and the triplet was inserted into the table, but the sending server either didn't try again, or tried again but used a different IP address, then the triplet would sit in the table until it expired.

If this is correct, then I can only see pending triplets, not ones that have been expired. This means that I have to keep a regular eye on this list, but keep a track of which entries in the table get marked as passed, and which are removed. That's going to be very difficult to track.

Isn't there some easier way of seeing which entries are removed?

Thanks
Alan

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: PLease help me reduce the spam - I'm not getting very far

Post by ^DooM^ » 2008-05-05 01:39

Oops my apologies, I have slept since then :mrgreen:

No easy way i can think of keeping a record of expired records without running a script on the triplets table hourly and updating a new table with required info. Not sure why you would need to see removed triplets though. 90% of them will be fake from addresses anyway.
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

AlanSilver
Normal user
Normal user
Posts: 190
Joined: 2007-04-22 17:06

Re: PLease help me reduce the spam - I'm not getting very far

Post by AlanSilver » 2008-05-05 14:35

^DooM^ wrote:Oops my apologies, I have slept since then :mrgreen:
Lucky you! I wish I had :D
^DooM^ wrote:No easy way i can think of keeping a record of expired records without running a script on the triplets table hourly and updating a new table with required info. Not sure why you would need to see removed triplets though. 90% of them will be fake from addresses anyway.
I know most will be rubbish, but it's the few genuine ones that I want to see. I'm slightly suspicious that not all of my mail is getting through, and I want to check. Seeing which records were expired would tell me which e-mails didn't get through. Assuming I could cope with the boredom of wading through all the duff ones, if I could see which genuine ones didn't get through, then I'd be able to assess the problem fully.

Thanks
Alan

User avatar
mattg
Moderator
Moderator
Posts: 20837
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: PLease help me reduce the spam - I'm not getting very far

Post by mattg » 2008-05-05 14:54

How about setting the 'days before removing unused records' very high, like months or years?

Matt
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

AlanSilver
Normal user
Normal user
Posts: 190
Joined: 2007-04-22 17:06

Re: PLease help me reduce the spam - I'm not getting very far

Post by AlanSilver » 2008-05-05 15:00

mattg wrote:How about setting the 'days before removing unused records' very high, like months or years?

Matt
Hmm, that's an idea. I could set it higher than the normal level, then check what's in there older than the normal level.

Thanks, I might try that for a short time and see what it shows.

Ta ra
Alan

User avatar
Slug
Moderator
Moderator
Posts: 1369
Joined: 2005-03-13 05:42
Location: Sydney Australia
Contact:

Re: PLease help me reduce the spam - I'm not getting very far

Post by Slug » 2008-05-05 16:41

I have my grey listing set to 6, 4, 60.

Works well for me.

Michael
Missing Hmailserver ... Now running Debian servers

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: PLease help me reduce the spam - I'm not getting very far

Post by ^DooM^ » 2008-05-05 16:49

7, 2, 72 here
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

AlanSilver
Normal user
Normal user
Posts: 190
Joined: 2007-04-22 17:06

Re: PLease help me reduce the spam - I'm not getting very far

Post by AlanSilver » 2008-05-05 17:09

Thanks Slug and Doom for that. Mine are set at 30, 1 and 36, which I think was the default. You both have longer days before removing unused records. Maybe I'll try increasing this and seeing if it helps. It's not a very busy server anyway, so the extra database records won't be a problem.

Thanks
Alan

User avatar
Slug
Moderator
Moderator
Posts: 1369
Joined: 2005-03-13 05:42
Location: Sydney Australia
Contact:

Re: PLease help me reduce the spam - I'm not getting very far

Post by Slug » 2008-05-05 17:09

^DooM^ wrote:7, 2, 72 here

I was using 2 here for a long while, but 1 isp here (optus) only retries every 24 hours and every now and again it would miss the boat... :lol:

Michael
Missing Hmailserver ... Now running Debian servers

User avatar
mattg
Moderator
Moderator
Posts: 20837
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: PLease help me reduce the spam - I'm not getting very far

Post by mattg » 2008-05-06 01:40

10,5,1000

subject to change of course.
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
Slug
Moderator
Moderator
Posts: 1369
Joined: 2005-03-13 05:42
Location: Sydney Australia
Contact:

Re: PLease help me reduce the spam - I'm not getting very far

Post by Slug » 2008-05-06 02:59

mattg wrote:10,5,1000
1000 --->> thats a big number young padawan :shock:
Missing Hmailserver ... Now running Debian servers

User avatar
mattg
Moderator
Moderator
Posts: 20837
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: PLease help me reduce the spam - I'm not getting very far

Post by mattg » 2008-05-06 04:02

Slug wrote:1000 --->> thats a big number young padawan :shock:
Is there a reason that I shouldn't use such a big number?

My aim was just to make the greylist only affect those that we don't get e-mail from regularly, 1000 is almost three years...

And, I am very happy to be called 'apprentice' from such a master Jedi as yourself :D

Thanks
Matt
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: PLease help me reduce the spam - I'm not getting very far

Post by ^DooM^ » 2008-05-06 10:09

Just keep an eye on your greylisting table size, it could potentially hit the 4gig limit and cause issues with such a high setting.
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

User avatar
mattg
Moderator
Moderator
Posts: 20837
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: PLease help me reduce the spam - I'm not getting very far

Post by mattg » 2008-05-06 13:32

Ahh Always learning from the masters. Good call - I will check on it.
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

gsmaclean
Normal user
Normal user
Posts: 47
Joined: 2008-04-16 18:35

Re: PLease help me reduce the spam - I'm not getting very far

Post by gsmaclean » 2008-05-06 16:04

I have to agree with the poster that suggested ASSP - I've been using it for a couple of years now, and it is by far the most effective spam killer I have ever used (and I've been doing this for over ten years now). The fact that it is free makes it all the much better. It is definitely not the kind of thing for a newbie to attempt to set up, however - you must invest a fair amount of time and effort learning it before you are able to implement it effectively.

My mail server has approximately 50,000-60,000 inbound emails a day, of which ASSP (correctly) filters out about 95% as spam. My users LOVE ASSP.

The only qualm I have with ASSP is that it does not scale all that well - with the load I have on it right now, it is at about the maximum it can handle. They are apparently working on a multi-threaded version that will solve this problem.

Post Reply