Greylisting and DNSBL - safe?

Use this forum if you want to discuss a problem or ask a question related to a hMailServer beta release.
Post Reply
Shadoxity
Normal user
Normal user
Posts: 97
Joined: 2006-01-06 06:20

Greylisting and DNSBL - safe?

Post by Shadoxity » 2006-11-05 12:48

Hey everyone,

I recently upgraded and enabled the above 2 to get rid of spam (working great btw :D)

Just wondering how safe it is to have these on? are they likely to get my real emails and delete them at all?

CHeers

User avatar
martin
Developer
Developer
Posts: 6837
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Post by martin » 2006-11-05 19:07

It's probably hard to give a good number. I don't think there's any 100% safe anti spam protection. Both grey listing and DNS bl will likely lead to that some legitimate email is lost. There's a large amount of different DNS blacklists as well. Some are high risk lists and some have lower risks..

Configure hMailServer to move all spam to a specific IMAP folder. Then you can run it for a while and see for yourself.

User avatar
dzekas
Senior user
Senior user
Posts: 2486
Joined: 2005-10-13 21:28
Location: Lithuania

Post by dzekas » 2006-11-05 19:25

Greylisting and RBL should not drop legit emails. If server is blocked by RBL, it should bounce email to email sender. Error message explains why delivery was blocked.

If delivery is delayed by greylisting, remote server must handle delays and resend the message. If misconfigured greylisting blocks delivery, email message will bounce to sender, when max spool time is reached.

User avatar
martin
Developer
Developer
Posts: 6837
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Post by martin » 2006-11-05 19:34

dzekas is right, the email isn't exactly "lost". The sender of the email should get a error message back saying that the email could not be delivered.

Shadoxity
Normal user
Normal user
Posts: 97
Joined: 2006-01-06 06:20

Post by Shadoxity » 2006-11-05 21:30

Ok,

So if an email gets removed due to the DNSBL then the user will just get an email saying, it didnt reach the intended recipient because...??

Is that right?

And what is RBL?

Cheers

User avatar
dzekas
Senior user
Senior user
Posts: 2486
Joined: 2005-10-13 21:28
Location: Lithuania

Post by dzekas » 2006-11-05 21:56

http://en.wikipedia.org/wiki/DNSBL

"A DNS-based Blackhole List (DNSBL, also known as Real-time Blackhole List or RBL)"

Same thing. Two different abbreviations.

RBL checks remote server in DNS based black lists. If server is in black list, it gets SMTP 5xx or 4xx error. 5xx error bounces message to sender. 4xx error delays delivery. Best way is to bounce email with 5xx error. Sender gets instant notification about delivery error. If email is legit and sender shows that error to his/her email admin, admin can work on resolving this issue.

Main RBL problem - finding blacklists that have low false positive rate. If RBL is too conservative - spammers are not blocked. If RBL is too aggressive - legit email is rejected.

Shadoxity
Normal user
Normal user
Posts: 97
Joined: 2006-01-06 06:20

Post by Shadoxity » 2006-11-05 22:02

ahhh.

Thanks for that.

How are the default hmailserver lists?

Can you recommend any?

cheers

Shadoxity
Normal user
Normal user
Posts: 97
Joined: 2006-01-06 06:20

Post by Shadoxity » 2006-11-05 23:24

How can i make it move the ones it finds as spam to a certain folder?

Cheers

User avatar
dzekas
Senior user
Senior user
Posts: 2486
Joined: 2005-10-13 21:28
Location: Lithuania

Post by dzekas » 2006-11-06 00:00

RBL and greylisting works at SMTP stage. rbl blocks server when remote server starts email delivery. greylisting delays delivery when service sees same remote address, sender's email and recipient email for the first time.

Greylisting does not block spam. It just delays delivery and forces spammer to maintain message queue. Most of spammers don't have resources to maintain queued messages. http://www.greylisting.org/articles/whitepaper.shtml. Own spammer's server can't process billions of emails. If they use provider's server, their account will be suspended. If they use hijacked bots, bots might be listed in dailup RBL. If they do use bots and machine tries to maintain message queue, it will be overloaded and user will take it to computer service.

RBL filtering blocks remote servers when they try to start message delivery. Usually server gets 5xx error on first smtp command. If server is blacklisted, it is stopped before message delivery and you don't have message copy.

If you want to use RBL and move message to spam folder, you should use RBL in tag only mode or use it with SpamAssassin or other content filtering software. Main purpose of RBL is to block spammer before he or she wastes your CPU, memory and disk space resources. If you tag and store RBLed emails, you are wasting your resources.

I can recommend spamhaus.org and cbl.abuseat.org RBLs. But they are conservative and you might need sorbs or njabl in order to block hijacked DSL hosts.

Please understand spam filtering technology which you are using.

Shadoxity
Normal user
Normal user
Posts: 97
Joined: 2006-01-06 06:20

Post by Shadoxity » 2006-11-06 00:13

Ahhhh thank you for that.

So it doesnt actually reach the mailserver to get deleted?

Thats good :D

User avatar
Slug
Moderator
Moderator
Posts: 1369
Joined: 2005-03-13 05:42
Location: Sydney Australia
Contact:

Post by Slug » 2006-11-06 13:34

The standard DNSBL that come with hMs are not very agressive, so its a good chance you will not lost any legit email. I am using about 6 DNSBL and I have lost about 3 email in about 18 months.

Michael
Missing Hmailserver ... Now running Debian servers

gbuktenica
New user
New user
Posts: 14
Joined: 2006-10-16 08:58
Contact:

Post by gbuktenica » 2006-11-17 03:48

My experience with greylisting is very positive with any RFC complient mail server. However a lot of web sites with forms that you fill in and then click a button to e-mail only run a dumb script and do not retry so the e-mail is lost.

AJB111
Normal user
Normal user
Posts: 184
Joined: 2005-01-28 05:13
Location: Australia
Contact:

Post by AJB111 » 2006-11-18 23:30

Hi Everyone

Just thought I would add my 2 cents worth to the Greylist experience.

Only been 4 days since implementing GreyListing and can report that 99.9% of SPAM has been stopped dead in it's tracks. In 4 days, only 2 reports of SPAM emails from my 5 largest accounts and no reports of mail not arriving.

To be truthfull I could have done this ages ago, but was nervous about loosing legit emails.

The worst delivery delay has been 2 hours, but once explaining in detail the reason to the recipient and offering to remove GreyListing, they opted to continue as they had NOT received any Spam since implementing it. When it comes down to it, we can not control that anyway, but a 2 hour re-try does seem a long time.

Are the Greylisting stats included in the Normal Stats? (Messages Containing Spam)

One thing I would like to see is seperate Stats on the results of Grey listing (not sure if this has been requested previously or is possibly implemented)

i.e:
Number of Emails Soft Rejected : 1000 (Initial Rejection)
Emails that Passed Successfully : 235 (Triplet True)
Possible Spam stopped by Greylisting : 765 (Difference between the two)

My own mail has gone from 60 - 100 spam messages a day to ZERO in the last 4 days.

Overall, the results have been outstanding (even for such a short time period) and the customers love it, so give it a go.

OK, these are my GreyListing Settings and would welcome comments from anyone who has been using it for a while as to if I have them right or some better settings - so far only my mail server IP address has been White listed ....

Image

These are the stats from less than a day (a weekend day at that) on the server.

Image

Martin, can not thank you enough for such a great product and the ultimate in support from you and the community you have created ... thanks again.
Last edited by AJB111 on 2006-11-19 23:51, edited 3 times in total.
Windows Server 2003, IIS6
HMail Server 5.3.1 B1748
MySQL 5.0.67

User avatar
martin
Developer
Developer
Posts: 6837
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Post by martin » 2006-11-18 23:34

Are the Greylisting stats included in the Normal Stats? (Messages Containing Spam)
No, not today. The reason for this is that it's impossible to measure the number of spam messages blocked by grey listing. When grey listing is used, hMailServer delays the delivery of an email. It's not possible for hMailServer to determine whether this email was spam or not. The only thing hMailServer can count is the number of accepts and blocks it makes based on grey listing.

AJB111
Normal user
Normal user
Posts: 184
Joined: 2005-01-28 05:13
Location: Australia
Contact:

Post by AJB111 » 2006-11-18 23:46

Thanks Martin

Even that number would be nice to know ... maybe I misused the word SPAM in my post.

Again, this has been very positive so far, I should have implemented it immediately.
Windows Server 2003, IIS6
HMail Server 5.3.1 B1748
MySQL 5.0.67

AJB111
Normal user
Normal user
Posts: 184
Joined: 2005-01-28 05:13
Location: Australia
Contact:

Post by AJB111 » 2006-11-19 23:53

Update:

Damn, maybe this is broken .... :lol: :wink:

Just received my FIRST Spam message in 5 Days !

What more can you ask for - does not get much better than this
Windows Server 2003, IIS6
HMail Server 5.3.1 B1748
MySQL 5.0.67

User avatar
DFitch
Senior user
Senior user
Posts: 258
Joined: 2006-09-16 20:40

Post by DFitch » 2006-11-20 00:01

Greylisting works very well, too well if you ask me. Blocks good bit legit too, unless you have a very good whitelist.

Boost
Normal user
Normal user
Posts: 52
Joined: 2006-01-05 00:45
Location: Denmark

Post by Boost » 2006-11-20 07:28

Greylisting dosnt block anything.
It only ask the sending mailserver to try again.

Most spam servers dosnt do that, but legit mail servers should definitely send the mail again at a later time.

User avatar
DFitch
Senior user
Senior user
Posts: 258
Joined: 2006-09-16 20:40

Post by DFitch » 2006-11-20 09:14

true, it rejects with 451 try again later, but unless whitelisted then it will continue to get rejected.

I use it, but have several rules in place.

if sender local and authenticated then disable greylisting
if sender address listed then disable greylisting
if sender Ip whitelisted(DB) or in optional whitelist(like yahoo groups, etc) then disable greylisting.

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Post by ^DooM^ » 2006-11-20 09:50

DFitch wrote:true, it rejects with 451 try again later, but unless whitelisted then it will continue to get rejected.
This is not true. As long as the triplet matches what is stored in the greylisting table when the email retries it will then be whitelisted internally for 36 days (Default).
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

ric melia
Normal user
Normal user
Posts: 33
Joined: 2004-12-13 15:11

Post by ric melia » 2006-11-20 11:01

gbuktenica wrote:However a lot of web sites with forms that you fill in and then click a button to e-mail only run a dumb script and do not retry so the e-mail is lost.
How much of an issue has this been?

Presumably you can get round this problem by clicking the button on the website again? (obviously undesirable though)

Can anyone think of any possible work arounds / solutions to this?

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Post by ^DooM^ » 2006-11-20 12:31

It is a small issue yes. I haven't yet had any issues. The way to get around it is whiltelist the email server :)

User avatar
martin
Developer
Developer
Posts: 6837
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Post by martin » 2006-11-20 13:07

However a lot of web sites with forms that you fill in and then click a button to e-mail only run a dumb script and do not retry so the e-mail is lost.
Is it really common that web sites works the way you describe it? All PHP/ASP scripts I have see delivers the email to a "real" local SMTP server. This local SMTP server then takes care of the delivery to the recipients server. I don't think I have ever seen a script which connects directly to the recipients server(s) to deliver a message.

User avatar
DFitch
Senior user
Senior user
Posts: 258
Joined: 2006-09-16 20:40

Post by DFitch » 2006-11-20 20:41

Understand the Triplet, but some resend and come in under another IP, then you get a Delay again, this is a problem, if it happens to some of my clients that email bids for jobs.

works well im just saying you will miss some legit mail as well.

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Post by ^DooM^ » 2006-11-20 21:10

Agreed. there is a whitelist that bazporter found/compiled to help eliminate some of the risk that you mention herewhich should help.

If you do get complaints then you can just whitelist their email servers too.

bazporter
Normal user
Normal user
Posts: 98
Joined: 2005-06-03 16:14

Post by bazporter » 2006-11-21 09:56

In a discussion on the SpamAssassin mailing list, the following statement was made:
You WILL have to add some IP's to a white list to not block braindead exchange (older versions) and Groupwise (lotus notes) servers that bounce on a 421 - Please try again later, instead of trying again later.
In the whitelist I put together I did have a bunch of Groupwise entries but it would be worth checking with key suppliers/customers that you work with if they use Groupwise so that you can add them to the whitelist straight away.

--
Regards
Barry

gbuktenica
New user
New user
Posts: 14
Joined: 2006-10-16 08:58
Contact:

Post by gbuktenica » 2006-11-27 05:13

martin wrote:
However a lot of web sites with forms that you fill in and then click a button to e-mail only run a dumb script and do not retry so the e-mail is lost.
Is it really common that web sites works the way you describe it? All PHP/ASP scripts I have see delivers the email to a "real" local SMTP server. This local SMTP server then takes care of the delivery to the recipients server. I don't think I have ever seen a script which connects directly to the recipients server(s) to deliver a message.
I don't think it is overly common, but I have seen it happen.

Post Reply