Is it possible to create an own DNS-Blacklist?

Use this forum if you have installed hMailServer and want to ask a question related to a production release of hMailServer. Before posting, please read the troubleshooting guide. A large part of all reported issues are already described in detail here.
Post Reply
User avatar
Rainer
Normal user
Normal user
Posts: 166
Joined: 2007-06-21 13:40
Location: Zweibrücken - Germany

Is it possible to create an own DNS-Blacklist?

Post by Rainer » 2008-04-04 10:42

Hello, is it possible to create an own DNS-Blacklist?
I wish to insert IP's found by extern in my own DNS-Blacklist.
The most IP's try to send very often and I think it's really stupid to check the extern DNS-Blacklists for IP's earlier found.

Regards :)
Rainer Noa

User avatar
SorenR
Senior user
Senior user
Posts: 3576
Joined: 2006-08-21 15:38
Location: Denmark

Re: Is it possible to create an own DNS-Blacklist?

Post by SorenR » 2008-04-04 11:23

SørenR.

“Those who don't know history are doomed to repeat it.”
― Edmund Burke

Carl
Normal user
Normal user
Posts: 111
Joined: 2008-03-18 17:33
Location: Utah, USA
Contact:

Re: Is it possible to create an own DNS-Blacklist?

Post by Carl » 2008-04-04 14:43

Yep. I have one, but unless you have lots of people to help maintain it, its easier to just rely on the larger lists.
Carl

98% of computer users know just enough to be dangerous, but not enough to be useful. I'm afraid I don't fall in the top 2%.

redrummy
Senior user
Senior user
Posts: 370
Joined: 2007-06-21 06:52
Location: Alaska

Re: Is it possible to create an own DNS-Blacklist?

Post by redrummy » 2008-04-04 18:20

I was going to say, if you're running an internal DNS server you can do whatever you want with it! Sounds like Carl's already on it.

Short version of Soren's link:
- Create blocked.mydomain zone
- Add A 4.3.2.1 = 127.0.0.2 in zone
- Add zone to hMS DNSBL
- Mail from 1.2.3.4 blocked by 4.3.2.1.blocked.mydomain

EDIT: Ryan was just skooled by Soren ;) (post updated)
Last edited by redrummy on 2008-04-04 18:32, edited 1 time in total.

User avatar
SorenR
Senior user
Senior user
Posts: 3576
Joined: 2006-08-21 15:38
Location: Denmark

Re: Is it possible to create an own DNS-Blacklist?

Post by SorenR » 2008-04-04 18:27

redrummy wrote:Mail from 1.2.3.4 blocked by 1.2.3.4.blocked.mydomain
No no...

Mail from 1.2.3.4 blocked by 4.3.2.1.blocked.mydomain

:wink:
SørenR.

“Those who don't know history are doomed to repeat it.”
― Edmund Burke

Carl
Normal user
Normal user
Posts: 111
Joined: 2008-03-18 17:33
Location: Utah, USA
Contact:

Re: Is it possible to create an own DNS-Blacklist?

Post by Carl » 2008-04-04 23:05

Yes, basically what you're doing is creating DNS entries that include the offending IP in reverse. So with my site, if I receive a spam email from 1.2.3.4 as the IP, I will create an entry for 4.3.2.1.rbl.skunkmanor.com When the server checks the dns records for that IP it comes back positive and hMS takes it from there. I've put many hours into it though and still only have about 500 sites blocked.
Carl

98% of computer users know just enough to be dangerous, but not enough to be useful. I'm afraid I don't fall in the top 2%.

User avatar
Rainer
Normal user
Normal user
Posts: 166
Joined: 2007-06-21 13:40
Location: Zweibrücken - Germany

Re: Is it possible to create an own DNS-Blacklist?

Post by Rainer » 2008-04-08 13:37

Carl wrote:Yes, basically what you're doing is creating DNS entries that include the offending IP in reverse. So with my site, if I receive a spam email from 1.2.3.4 as the IP, I will create an entry for 4.3.2.1.rbl.skunkmanor.com When the server checks the dns records for that IP it comes back positive and hMS takes it from there. I've put many hours into it though and still only have about 500 sites blocked.
Hello Carl, my wish is to have a own blocklist looking by spamhaus and the other DNSBL's.
When a ip is found, the ip should be added to my DNSBL.
In hMailServer I need only my own DNSBL.

Is this possible?
Rainer Noa

Carl
Normal user
Normal user
Posts: 111
Joined: 2008-03-18 17:33
Location: Utah, USA
Contact:

Re: Is it possible to create an own DNS-Blacklist?

Post by Carl » 2008-04-08 16:07

With a lot of scripting and programming, possibly, if you're running your own DNS server. In my case, the best I could do (if I could script) would be take the information provided by a log parsing script and add it manually to my own DNS records, since the DNS server I use is not mine.

If you're going to be pulling the information from those 2 rbl's though, you might as well just add the address of the ones that get through anyway.
Carl

98% of computer users know just enough to be dangerous, but not enough to be useful. I'm afraid I don't fall in the top 2%.

phil54
Normal user
Normal user
Posts: 195
Joined: 2007-11-26 13:13
Location: UK :-)

Re: Is it possible to create an own DNS-Blacklist?

Post by phil54 » 2008-04-08 16:31

Out of interest and sorry for hijacking this thread rainer, i've got three blacklists running at the moment:

zen.spamhaus.org
bl.spamcop.net
dnsbl.njabl.org

After a lot of help from Soren with the sql queries in logparser, i can see that spamhaus stops the majority of the spam on the server, with the other two stopping a couple each.

Question 1) Is it worth having the other two ?
Question 2) Is it worth using this lot below, as well?

list.dsbl.org
opm.blitzed.org
psbl.surriel.com

Edit: I've just come across this site:

http://www.sdsc.edu/~jeff/spam/2008/bc-20080322.html

I notice this set of blacklists seem to be doing quite well, anyone use them with hmail server before?

t1.dnsbl.net.au
cbl.abuseat.org
dnsbl.sorbs.net
l2.apews.org

Thanks all
Image

redrummy
Senior user
Senior user
Posts: 370
Joined: 2007-06-21 06:52
Location: Alaska

Re: Is it possible to create an own DNS-Blacklist?

Post by redrummy » 2008-04-08 18:44

phil54,
Thanks for the link, that's an interesting list. I've been using dnsbl.sorbs.net (and zen.spamhaus.org) for quite a while and do pretty well with them.
My 2c...

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: Is it possible to create an own DNS-Blacklist?

Post by ^DooM^ » 2008-04-08 20:19

I don't use SORBS for this reason

http://www.lazyllama.com/blog/archives/ ... orbs-sucks

Dunno if they are still charging to be removed but even so I still wouldn't use them even if they aren't any more.
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

Carl
Normal user
Normal user
Posts: 111
Joined: 2008-03-18 17:33
Location: Utah, USA
Contact:

Re: Is it possible to create an own DNS-Blacklist?

Post by Carl » 2008-04-08 21:51

To me, if one blocks a single message once a year that the others don't, it's worth keeping the rbl running on hMS for me. A higher volume server may not feel the same way.

The main reason for my maintaining my own rbl is that none of those listed do a very good job of tracking spam in the US. At least not the stuff that I get. Most of the ones on my site are Russian and American.
Carl

98% of computer users know just enough to be dangerous, but not enough to be useful. I'm afraid I don't fall in the top 2%.

redrummy
Senior user
Senior user
Posts: 370
Joined: 2007-06-21 06:52
Location: Alaska

Re: Is it possible to create an own DNS-Blacklist?

Post by redrummy » 2008-04-09 05:54

I don't use SORBS for this reason

http://www.lazyllama.com/blog/archives/ ... orbs-sucks

Dunno if they are still charging to be removed but even so I still wouldn't use them even if they aren't any more.
I'll admit that SORBS can be a bit... aggressive, but it's policies are well defined and fair, IMHO. That link is a bit of a rant and gives the impression that ALL de-listings require payment. The de-listing process is usually automatic and free. An address can be listed for any number of reasons, but only certain confirmed spam sources are charged a "fine" to be de-listed. I've never encountered an address that I wansted to de-list that wan't free. My 2c...

User avatar
Rainer
Normal user
Normal user
Posts: 166
Joined: 2007-06-21 13:40
Location: Zweibrücken - Germany

Re: Is it possible to create an own DNS-Blacklist?

Post by Rainer » 2008-04-09 07:59

Hello, my Blacklists are these:
hMailAdminSnip.JPG
hMailAdminSnip.JPG (22.81 KiB) Viewed 12610 times
The german list NixSpam (http://www.dnsbl.manitu.net/?language=en)is really one of the best DNSBL 8)
The cn.countries.nerd.dk list is to block :evil: China :evil: --> Rejected because China - No violence - One World - One Dream - Free Tibet... :evil:
All the other DNSBL are good too and I'm happy to use this tools, because I don't need ASSP or SpamAssassin.
All DNSBL are trusted to reject the email.

The new option in version 5 to end the spam-checks when spam-score is reached is really needed.
Really nice/needed is the feature request (http://www.hmailserver.com/forum/viewto ... =2&t=11511) to create a own ip based blocklist to save bandwitdh and ressources.

Kind regards :)
Rainer Noa

phil54
Normal user
Normal user
Posts: 195
Joined: 2007-11-26 13:13
Location: UK :-)

Re: Is it possible to create an own DNS-Blacklist?

Post by phil54 » 2008-04-09 11:42

If a mail comes in, is it checked against the first blacklist, then the second, third etc.
Image

User avatar
Rainer
Normal user
Normal user
Posts: 166
Joined: 2007-06-21 13:40
Location: Zweibrücken - Germany

Re: Is it possible to create an own DNS-Blacklist?

Post by Rainer » 2008-04-09 11:55

phil54 wrote:If a mail comes in, is it checked against the first blacklist, then the second, third etc.
Hello Phil54, Yes it is!
Better is, mail comes in and checked the first blacklist, if it found within the 1st check, the other blacklists-checks stopped and email will rejected.

In the new version 5.x this is released by scoring.
I think there is are options like this:
1. "Reject email spam-score reached: nnn" --> here you can enter 100 (example)
2. You can enter for every Spam-Feature a score (example: Spamhaus: 100, NixSpam: 75, Surriel: 25, HELO-Check: 75, SPF: 100, MX-Record: 100 etc.)

Kind regards :)
Rainer Noa

phil54
Normal user
Normal user
Posts: 195
Joined: 2007-11-26 13:13
Location: UK :-)

Re: Is it possible to create an own DNS-Blacklist?

Post by phil54 » 2008-04-09 11:56

Excellent thanks rainer, i'll add another couple of blacklists later and monitor how many get rejected
Image

User avatar
Rainer
Normal user
Normal user
Posts: 166
Joined: 2007-06-21 13:40
Location: Zweibrücken - Germany

Re: Is it possible to create an own DNS-Blacklist?

Post by Rainer » 2008-04-09 12:16

phil54 wrote:Excellent thanks rainer, i'll add another couple of blacklists later and monitor how many get rejected
Hello Phil, be carefull with use of not trusted DNSBL's.
Some blocking all of the mass-mailing-provider.
Some want for unblocking green-bucks.
And some's blocking-policy is a mystery.

Regards :)
Rainer Noa

phil54
Normal user
Normal user
Posts: 195
Joined: 2007-11-26 13:13
Location: UK :-)

Re: Is it possible to create an own DNS-Blacklist?

Post by phil54 » 2008-04-09 12:23

haha thanks rainer, i'm using:

zen.spamhaus.org
bl.spamcop.net
dnsbl.njabl.org

and i think i'll add:

psbl.surriel.com
ix.dnsbl.manitu.net

and see how i go from there :D
Image

phil54
Normal user
Normal user
Posts: 195
Joined: 2007-11-26 13:13
Location: UK :-)

Re: Is it possible to create an own DNS-Blacklist?

Post by phil54 » 2008-04-23 11:47

How many people use the psbl.surriel.com blacklist? What sort of results are you getting from it?

Reason i'm asking, i added it in around the start of this month and so far it hasnt blocked 1 spam mail. Were i added the manitu list last week and it seems to be blocking quite a few everyday.

This is what i'm using at the moment, any others i should add or remove from this list?

zen.spamhaus.org
bl.spamcop.net
dnsbl.njabl.org
psbl.surriel.com
ix.dnsbl.manitu.net
Image

Carl
Normal user
Normal user
Posts: 111
Joined: 2008-03-18 17:33
Location: Utah, USA
Contact:

Re: Is it possible to create an own DNS-Blacklist?

Post by Carl » 2008-04-23 16:59

Never heard of that one.

Do you use a log parser to track what is rejecting what? I'm trying to find a good way to check what lists are rejecting anything. So far I do a "Find" on the word "rejected" but there's got to be a more automated way.
Carl

98% of computer users know just enough to be dangerous, but not enough to be useful. I'm afraid I don't fall in the top 2%.

redrummy
Senior user
Senior user
Posts: 370
Joined: 2007-06-21 06:52
Location: Alaska

Re: Is it possible to create an own DNS-Blacklist?

Post by redrummy » 2008-04-23 19:06

Can probably do this w/ MS Log Parser... Haven't used it personally yet, but there was QUITE a discussion about it here: http://www.hmailserver.com/forum/viewto ... =7&t=11792

Carl
Normal user
Normal user
Posts: 111
Joined: 2008-03-18 17:33
Location: Utah, USA
Contact:

Re: Is it possible to create an own DNS-Blacklist?

Post by Carl » 2008-04-24 02:11

Yeah.

Well. I couldn't follow it. :P
Carl

98% of computer users know just enough to be dangerous, but not enough to be useful. I'm afraid I don't fall in the top 2%.

phil54
Normal user
Normal user
Posts: 195
Joined: 2007-11-26 13:13
Location: UK :-)

Re: Is it possible to create an own DNS-Blacklist?

Post by phil54 » 2008-04-24 10:44

Hi Carl,
yeah i've been using log parser to do this, with a lot of help from Soren.

Easy to do:

1) Create a file called "smtpd.tsv with the following inside"
logtype timestamp remotehost text excess (Tab between each)

2) Download logparser from the ms site

3) Create a batch file and put the following code in, stick it in the logparser directory along with the hmail log file, you'll need to change mail.log to whatever.log

echo off
echo ** 550 Rejected by Manitu **
LogParser "SELECT * FROM mail.log TO MANITU.csv WHERE logtype LIKE '_SMTPD_' AND excess LIKE '%%SENT: 550 Rejected by Manitu%%'" -i:TSV -headerRow OFF -iHeaderFile smtpd.tsv -iSeparator TAB

This should give you all the mails rejected by that particular list, i'm sure you could put them all together in one query but i'm happy it's just working at the moment :mrgreen:
Image

phil54
Normal user
Normal user
Posts: 195
Joined: 2007-11-26 13:13
Location: UK :-)

Re: Is it possible to create an own DNS-Blacklist?

Post by phil54 » 2008-04-24 11:57

Does anyone use this one?

http://cbl.abuseat.org/
Image

User avatar
dzekas
Senior user
Senior user
Posts: 2486
Joined: 2005-10-13 21:28
Location: Lithuania

Re: Is it possible to create an own DNS-Blacklist?

Post by dzekas » 2008-04-24 12:04

phil54 wrote:Does anyone use this one?

http://cbl.abuseat.org/
You do. With your zen.spamhaus.org line.

phil54
Normal user
Normal user
Posts: 195
Joined: 2007-11-26 13:13
Location: UK :-)

Re: Is it possible to create an own DNS-Blacklist?

Post by phil54 » 2008-04-24 12:13

well i didnt know that, thanks dzekas :D
Image

User avatar
Rainer
Normal user
Normal user
Posts: 166
Joined: 2007-06-21 13:40
Location: Zweibrücken - Germany

Re: Is it possible to create an own DNS-Blacklist?

Post by Rainer » 2008-04-24 12:59

Her are the spam-statistics from Tuesday:

SmtpText----------------------------------------------------------------------, Count -----------------
550 The host name specified in HELO does not match IP address, 3300
550 Rejected because black-listed at DNSBL: ix.dnsbl.manitu.net, 1669
550 Rejected because black-listed at DNSBL: zen.spamhaus.org, 1436
550 Rejected because China - No violence - One World - One Dream - Free Tibet, 330
550 Blocked by SPF, 212
550 Domain does not have any MX records, 165
550 Rejected because black-listed at DNSBL: safe.dnsbl.sorbs.net, 27
550 Rejected because black-listed at DNSBL: psbl.surriel.com, 15
550 Rejected because black-listed at DNSBL: bl.spamcop.net, 9
550 Unknown user, 7
550 The address is not valid, 2

Kind regards :)
Rainer Noa

phil54
Normal user
Normal user
Posts: 195
Joined: 2007-11-26 13:13
Location: UK :-)

Re: Is it possible to create an own DNS-Blacklist?

Post by phil54 » 2008-04-24 14:07

nice going rainer, is that the output from one sql query?
Image

User avatar
Rainer
Normal user
Normal user
Posts: 166
Joined: 2007-06-21 13:40
Location: Zweibrücken - Germany

Re: Is it possible to create an own DNS-Blacklist?

Post by Rainer » 2008-04-24 14:35

Hello Phil, take a look to the attached zip-file.
There are 2 files.
One is the definition for the hMailServer-Log and the cmd-file is for reporting.

Kind regards :)
Attachments
LogParser.zip
(560 Bytes) Downloaded 172 times
Rainer Noa

phil54
Normal user
Normal user
Posts: 195
Joined: 2007-11-26 13:13
Location: UK :-)

Re: Is it possible to create an own DNS-Blacklist?

Post by phil54 » 2008-04-24 15:14

Thanks Rainer, thats excellent. It just shows you can do a lot with this program if you know what your doing :mrgreen:

I wonder if it would be possible to create one that will tell you how many spam emails are getting through? I think i'll have a look into this
Image

rodolfor
Senior user
Senior user
Posts: 282
Joined: 2005-06-30 09:05
Location: Gubbio - Italy

Re: Is it possible to create an own DNS-Blacklist?

Post by rodolfor » 2008-04-25 10:40

I think that the rainer list is not reliable because it depend from the order of blacklist.
If zen.spamhouse being the first in the list, the above ranking coud be very differente.
For example, I have cbl.abuseat.org as first, then combined.njabl.org and bl.spamcop.net. Over there I have multi.surbl.org.
I dont use 'Check host in helo command' because of false positives.
My statistics are (from 1st april):

118679 Rejected by cbl.abuseat.org
6870 Rejected by bl.spamcop.net
9359 Unknown user
1685 Domain does not have any MX records
22 The address is not valid

It seems that: combined.njabl.org and multi.surbl.org does not affect results.
Hmailserver [lastversion] + MSSQL

User avatar
dzekas
Senior user
Senior user
Posts: 2486
Joined: 2005-10-13 21:28
Location: Lithuania

Re: Is it possible to create an own DNS-Blacklist?

Post by dzekas » 2008-04-25 10:51

rodolfor wrote: 6870 Rejected by bl.spamcop.net
Spamcop docs(1, 2) recommend use of blacklist only for tagging.

User avatar
Rainer
Normal user
Normal user
Posts: 166
Joined: 2007-06-21 13:40
Location: Zweibrücken - Germany

Re: Is it possible to create an own DNS-Blacklist?

Post by Rainer » 2008-04-25 11:10

Hello, yes it's right!

The amount of the hits is depending on the order in hMailServer.
If spamhaus is the first DNSBL so spamhaus is the leader of the charts! :)
NixSpam is on my list the first DNSBL because NixSpam answers very fast.

Greetings and have a nice weekeend :)
Rainer Noa

phil54
Normal user
Normal user
Posts: 195
Joined: 2007-11-26 13:13
Location: UK :-)

Re: Is it possible to create an own DNS-Blacklist?

Post by phil54 » 2008-04-28 21:57

With my crappy sql skills, i've managed to get a list of all mails going to the mail server and another list of all the mails being rejected, what i cant do is combine them. :evil:

I want the above two and then another list of mails that have been accepted, anyone got any ideas? Ie Soren and Rainer :mrgreen:
Image

User avatar
SorenR
Senior user
Senior user
Posts: 3576
Joined: 2006-08-21 15:38
Location: Denmark

Re: Is it possible to create an own DNS-Blacklist?

Post by SorenR » 2008-04-28 22:54

phil54 wrote:With my crappy sql skills, i've managed to get a list of all mails going to the mail server and another list of all the mails being rejected, what i cant do is combine them. :evil:

I want the above two and then another list of mails that have been accepted, anyone got any ideas? Ie Soren and Rainer :mrgreen:
I would put my money on Rainer :wink:
SørenR.

“Those who don't know history are doomed to repeat it.”
― Edmund Burke

User avatar
Rainer
Normal user
Normal user
Posts: 166
Joined: 2007-06-21 13:40
Location: Zweibrücken - Germany

Re: Is it possible to create an own DNS-Blacklist?

Post by Rainer » 2008-04-29 07:30

Hello Phil, I think the log-parsers SQL-Implementaion is not fully the same as a really SQL-Engine like SQL-Server or MySQL.
In the standard SQL there is the possibility to Join 2 or more query to one.
The syntax is this: Select * from x UNION Select * from y.
The queries must have the same field-names, the same data-types and the same amount of fields.
It's a really good command.

I'm a little bit in stress with my projects here and maybe there is next month a time-window to look on your SQL-Query.
SQL is in my opinion really easy to learn.
Try and error is the best practice here. :wink:

Kind regards :)
Rainer Noa

phil54
Normal user
Normal user
Posts: 195
Joined: 2007-11-26 13:13
Location: UK :-)

Re: Is it possible to create an own DNS-Blacklist?

Post by phil54 » 2008-04-29 10:43

thanks rainer, i'll have a play about with that command today
Image

Post Reply