False positive SURBL match

Use this forum if you have installed hMailServer and want to ask a question related to a production release of hMailServer. Before posting, please read the troubleshooting guide. A large part of all reported issues are already described in detail here.
Post Reply
User avatar
Nime
Normal user
Normal user
Posts: 116
Joined: 2009-03-12 11:50
Contact:

False positive SURBL match

Post by Nime » 2019-12-16 16:38

I've disabled SURBL server (multi.surbl.org) bec. of false positives. Below is the log snippet:
"DEBUG" 3548 "2019-12-16 09:14:20.303" "Adding task AsynchronousTask to work queue Asynchronous task queue"
"DEBUG" 3284 "2019-12-16 09:14:20.303" "Executing task AsynchronousTask in work queue Asynchronous task queue"
"DEBUG" 3284 "2019-12-16 09:14:20.303" "SURBL: Execute"
"DEBUG" 3284 "2019-12-16 09:14:20.303" "SURBL: Found URL: microsoft.com"
"DEBUG" 3284 "2019-12-16 09:14:20.303" "SURBL: Found URL: w3.org"
"DEBUG" 3284 "2019-12-16 09:14:20.303" "SURBL: 2 unique addresses found."
"DEBUG" 3284 "2019-12-16 09:14:20.319" "SURBL: Lookup: microsoft.com.multi.surbl.org"
"DEBUG" 3284 "2019-12-16 09:14:20.319" "SURBL: Match found"
"DEBUG" 3284 "2019-12-16 09:14:20.319" "Spam test: SpamTestSURBL, Score: 3"
"DEBUG" 3284 "2019-12-16 09:14:20.319" "Total spam score: 3"
"SMTPD" 3284 285466 "2019-12-16 09:14:20.319" "209.85.208.178" "SENT: 554 Rejected by SURBL."
"APPLICATION" 3284 "2019-12-16 09:14:20.319" "hMailServer SpamProtection rejected RCPT (Sender: muhasebe1@gokcuoglu.com, IP:209.85.208.178, Reason: Rejected by SURBL.)"
"DEBUG" 3284 "2019-12-16 09:14:20.319" "Deleting message file."
As you see the two safe url match somehow and the message has been rejected & deleted. What's wrong with SURBL? Did anyone notice?

P.S: The web query returns no false positive:
http://www.surbl.org/surbl-analysis
microsoft.com.multi.surbl.org is NOT listed

palinka
Senior user
Senior user
Posts: 1562
Joined: 2017-09-12 17:57

Re: False positive SURBL match

Post by palinka » 2019-12-16 17:44

This is strange. You're not the first person to report this issue.

https://www.hmailserver.com/forum/viewt ... 97#p216697

User avatar
mattg
Moderator
Moderator
Posts: 20554
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: False positive SURBL match

Post by mattg » 2019-12-16 23:22

What DNS are you using?
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

palinka
Senior user
Senior user
Posts: 1562
Joined: 2017-09-12 17:57

Re: False positive SURBL match

Post by palinka » 2019-12-16 23:31

https://www.spamhaus.org/faq/section/Spamhaus%20DBL#330
I am in China. Why is DBL listing non-spam domains such as twitter.com, facebook.com or pinterest.com ?

The DBL is not listing twitter.com, facebook.com, pinterest.com or other social network domains. However, as part of the Golden Shield Project (most commonly known as the Great FireWall of China) operated by the Ministry of Public Security (MPS) division of the government of China, DNS packets entering or exiting China can be altered by the Great FireWall if they contain particular keywords or domains.

As it turns out, these filters are implemented in a very crude and rudimentary way which may alter answers to Spamhaus DBL queries that are crossing the firewall (in both directions). So, for instance, our servers answer NXDOMAIN (meaning "this domain is not listed") to queries for twitter.com.dbl.spamhaus.org, but this answer may become something like

Code: Select all

        twitter.com.dbl.spamhaus.org. IN	A	159.106.121.75
upon crossing the China Internet boundary. Any software interpreting the presence of an A record as a signal that the domain is listed by DBL would then block mail from twitter.com in error.
This interference of the Chinese government's system with the operation of the Spamhaus infrastructure has the following consequences:

Spamhaus has servers in China to better serve the Chinese user base, but those servers can not be used to serve the DBL, to prevent disservices to users located out of China that happen to query them. So they are only used to answers queries relative to IP addresses (SBL, PBL, XBL).

Spamhaus users in China will get all DBL answers from servers located outside China, and therefore answers can be altered as described above. It is therefore fundamental that all users in China validate our responses by having their software check that the A record is a valid one in the range 127.0.1.0-127.0.1.255. Any other code is a Great Firewall artifact and in this case it must be assumed that the queried domain is not listed by DBL.
This could be a clue. The point being that *any* return on surbl is viewed as a hit, so if DNS is returning an IP (instead of a return code), then something is definitely wrong with your DNS.

Perhaps DNS was hijacked. If you change windows DNS settings to google or opendns and that resolves the problem, then you know your router was DNS hijacked.

User avatar
mattg
Moderator
Moderator
Posts: 20554
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: False positive SURBL match

Post by mattg » 2019-12-16 23:38

And in that post the great firewall of China will have an impact
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

palinka
Senior user
Senior user
Posts: 1562
Joined: 2017-09-12 17:57

Re: False positive SURBL match

Post by palinka » 2019-12-17 00:14

And the chinese are notorious for backdoors in hardware they build. :mrgreen:

User avatar
mattg
Moderator
Moderator
Posts: 20554
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: False positive SURBL match

Post by mattg » 2019-12-17 01:49

As are many other countries, including yours my friend

Software too, not just hardware
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

palinka
Senior user
Senior user
Posts: 1562
Joined: 2017-09-12 17:57

Re: False positive SURBL match

Post by palinka » 2019-12-17 03:55

Absolutely agree. Not making any judgements, just a statement of fact.

tunis
Normal user
Normal user
Posts: 241
Joined: 2015-01-05 20:22
Location: Sweden

Re: False positive SURBL match

Post by tunis » 2019-12-17 09:23

I has also got false hits with multi.surbl.org.
But no false hit with dbl.spamhaus.org.
HMS 5.6.8 B2494.24 on Windows Server 2019 Core VM.
HMS 5.6.8 B2494.24 on Windows Server 2016 Core VM.
HMS 5.6.7 B2425.16 on Windows Server 2012 R2 Core VM.

User avatar
Nime
Normal user
Normal user
Posts: 116
Joined: 2009-03-12 11:50
Contact:

Re: False positive SURBL match

Post by Nime » 2019-12-17 16:55

I'm not from China.
My DNS is 127.0.0.1 / WAN IP: 195.174.32.234
Is dbl.spamhaus.org alternative to multi.surbl.org?

palinka
Senior user
Senior user
Posts: 1562
Joined: 2017-09-12 17:57

Re: False positive SURBL match

Post by palinka » 2019-12-17 17:30

Nime wrote:
2019-12-17 16:55
I'm not from China.
My DNS is 127.0.0.1 / WAN IP: 195.174.32.234
Is dbl.spamhaus.org alternative to multi.surbl.org?
You don't have to be IN China to have your DNS routed TRHOUGH China. Anyway, the point was not that anyone having this issue is in China, but just to point out that your DNS is obviously messed up and it could be for many reasons.

Try this:

Open cmd window and type "nslookup microsoft.com.multi.surble.org". The response should be " can't find microsoft.com.multi.surble.org: Non-existent domain".

If you get ANY OTHER answer that will trigger hmailserver to see the lookup as a positive result. A positive result from multi.surbl.org will be a dns response code like: 127.0.0.64

If you get a regular IP address, which is perhaps the IP of the domain you're checking - that means something is WRONG. The response from multi.surbl.org should ONLY BE a 127.0.0.x response code or NOTHING (NXDOMAIN). Anything else is a problem with your DNS.

From their FAQ:
I'm using an anti-spam or anti-phishing DNS proxy, and I'm seeing legitimate sites marked as unsolicited.

There are some DNS proxy or modification services that change the responses from certain DNS queries in order to prevent users from visiting sites advertised in phishing, unsolicited messages, etc. This can cause errors when using SURBLs if the proxies return an IP address of an alternative (safe) web site. The modified IP address can have an incorrect effect on SURBL list identification depending on where the bit patterns happen to be in the modified response. The result is that legitimate sites may be misidentified, but the effect appears to be somewhat random or arbitrary.

A solution is to disable such site correction or modification features on servers or clients doing SURBL queries. Alternatively, consider using regular (non-modifying) nameservers for those systems. Often the best solution is to set up a local caching nameserver.

Note also that SURBL applications may be incompatible with DNS modification or proxy services that change the DNS query results of non-matches (NXDOMAIN results) for non-existent sites.

Note that as of 1/25/07, OpenDNS no longer modifies results for SURBL lists. It should now be safe to use OpenDNS with SURBL applications. If you find you are behind a firewall or proxy that is modifying SURBL DNS queries incorrectly, one solution is to set up a local caching nameserver. A local caching server can significantly improve performance also.

I'm using my provider's nameservers, and I'm seeing legitimate sites marked as unsolicited.

Some ISPs such as Verizon and Charter are reportedly modifying some DNS NXDOMAIN responses in a way that causes what look like false positives on domains that are not blacklisted. Unfortunately this breaks DNS responses for SURBLs and other blacklists. Please check with your ISP if you are seeing DNS responses modified in this way. Verizon has an opt-out procedure with instructions on switching to DNS servers that do not change NXDOMAIN responses. Others such as Charter have opt-out nameservers that reportedly do not support NXDOMAIN, in which case none of their nameservers may be compatible with DNS blacklists. One solution is to not use your provider's nameservers, for example by setting up your own local caching nameserver instead. Most operating systems have built-in support for running your own nameservers, and a local nameserver can significantly improve performance.
Based on the above, your problem could be your ISP's DNS is faulty, or the router is misconfigured or hijacked. Either way, the solution is to get DNS back onto your server (bypass your router and ISP for DNS).

On your server, open networking > your NIC properties > Internet Protocol Version 4 properties > manually set DNS option to either opendns (208.67.222.222, 208.67.220.220) or google dns (8.8.8.8, 8.8.4.4). I don't remember if a restart is required.

THEN open cmd prompt > "nslookup microsoft.com.multi.surble.org" and see if you get the correct response (NOTHING).

palinka
Senior user
Senior user
Posts: 1562
Joined: 2017-09-12 17:57

Re: False positive SURBL match

Post by palinka » 2019-12-17 17:31

Nime wrote:
2019-12-17 16:55
Is dbl.spamhaus.org alternative to multi.surbl.org?
Yes, but you still need to fix your broken DNS.

User avatar
Nime
Normal user
Normal user
Posts: 116
Joined: 2009-03-12 11:50
Contact:

Re: False positive SURBL match

Post by Nime » 2019-12-18 12:26


C:\Users\ea>nslookup
Default Server: 213-142-152-254.reverse.adeox.net
Address: 213.142.152.254

> microsoft.com.multi.surbl.org
Server: 213-142-152-254.reverse.adeox.net
Address: 213.142.152.254

*** 213-142-152-254.reverse.adeox.net can't find microsoft.com.multi.surbl.org: Non-existent domain
> server 8.8.8.8
Default Server: dns.google
Address: 8.8.8.8

> microsoft.com.multi.surbl.org
Server: dns.google
Address: 8.8.8.8

*** dns.google can't find microsoft.com.multi.surbl.org: Non-existent domain
>


User avatar
Nime
Normal user
Normal user
Posts: 116
Joined: 2009-03-12 11:50
Contact:

Re: False positive SURBL match

Post by Nime » 2019-12-18 13:36

After setting up a different DNS server it looks like fixed:
"DEBUG" 3724 "2019-12-18 14:12:41.344" "Adding task AsynchronousTask to work queue Asynchronous task queue"
"DEBUG" 3440 "2019-12-18 14:12:41.344" "Executing task AsynchronousTask in work queue Asynchronous task queue"
"DEBUG" 3440 "2019-12-18 14:12:41.344" "SURBL: Execute"
"DEBUG" 3440 "2019-12-18 14:12:41.344" "SURBL: Found URL: microsoft.com"
"DEBUG" 3440 "2019-12-18 14:12:41.344" "SURBL: Found URL: w3.org"
"DEBUG" 3440 "2019-12-18 14:12:41.344" "SURBL: 2 unique addresses found."
"DEBUG" 3440 "2019-12-18 14:12:41.344" "SURBL: Lookup: microsoft.com.multi.surbl.org"
"DEBUG" 3440 "2019-12-18 14:12:41.437" "SURBL: Lookup: w3.org.multi.surbl.org"
"DEBUG" 3440 "2019-12-18 14:12:41.453" "SURBL: Match not found"
"DEBUG" 3440 "2019-12-18 14:12:41.453" "Spam test: SpamTestSURBL, Score: 0"

palinka
Senior user
Senior user
Posts: 1562
Joined: 2017-09-12 17:57

Re: False positive SURBL match

Post by palinka » 2019-12-18 13:39

👍

Meister
Normal user
Normal user
Posts: 79
Joined: 2005-11-10 16:48

Re: False positive SURBL match

Post by Meister » 2019-12-18 14:29

I have used 1.1.1.1 and 1.0.0.1 as NS, and it sends back 127.0.0.1 for all non-existing domain. :-(
So I changed to 4.2.2.1 and 4.2.2.2 and the SURBL works again.

Oh, and I am opening a support ticket at Cloudflare to change the behavior of this domain in their NS.

Meister
Normal user
Normal user
Posts: 79
Joined: 2005-11-10 16:48

Re: False positive SURBL match

Post by Meister » 2019-12-18 14:52

I found this answer:
https://community.cloudflare.com/t/doma ... 0-1/130083

So seems to me, surbl.org cannot be reached via 1.1.1.1 NS.

But I don't understand then, why 4.2.2.2 works. But it is offtopic, but to log, the solution if surbl doesn't work, change your nameserver to other, and try again.

User avatar
jimimaseye
Moderator
Moderator
Posts: 8309
Joined: 2011-09-08 17:48

Re: False positive SURBL match

Post by jimimaseye » 2019-12-18 15:24

This is intended and correct behaviour, do not use public resolvers to query DNSBL services that have usage policies
because it is not a global public nameserver presumably being isp controlled.
This IP space is currently run by Level 3 (headquartered just down the road from us in Broomfield), and actually is a large number of machines. These machines are spread out over Level 3's network and your closest is located by a mechanism called "Anycast".
https://www.tummy.com/articles/famous-dns-server/

[Entered by mobile. Excuse my spelling.]
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
SorenR
Senior user
Senior user
Posts: 3328
Joined: 2006-08-21 15:38
Location: Denmark

Re: False positive SURBL match

Post by SorenR » 2019-12-19 04:37

palinka wrote:
2019-12-17 00:14
And the chinese are notorious for backdoors in hardware they build. :mrgreen:
Just remember for every finger you point at anything or anyone there are three fingers pointing your way ...

If you are right about the backdoors then they are a lot smarter than the americans - they don't get caught and they don't "go big" ... like ... er ... Stuxnet ... :roll:

Oh and in recent times NSA's been evesdropping on Angela Merkel's phone ... Nasty business this spying on state leaders by foreign goverments ... *cough* *cough* Huawei ban *cough*

1968 Thule Air Base B-52 crash ... Denmark is and was at the time a NUKE FREE zone. Yet there are four nukes burried in the ice near Thule Air Base ... Greenland ... Danish territory ... Not for sale!
SørenR.

“With age comes wisdom, but sometimes age comes alone.”
- Oscar Wilde

palinka
Senior user
Senior user
Posts: 1562
Joined: 2017-09-12 17:57

Re: False positive SURBL match

Post by palinka » 2019-12-19 04:42

SorenR wrote:
2019-12-19 04:37
palinka wrote:
2019-12-17 00:14
And the chinese are notorious for backdoors in hardware they build. :mrgreen:
Just remember for every finger you point at anything or anyone there are three fingers pointing your way ...

If you are right about the backdoors then they are a lot smarter than the americans - they don't get caught and they don't "go big" ... like ... er ... Stuxnet ... :roll:

Oh and in recent times NSA's been evesdropping on Angela Merkel's phone ... Nasty business this spying on state leaders by foreign goverments ... *cough* *cough* Huawei ban *cough*

1968 Thule Air Base B-52 crash ... Denmark is and was at the time a NUKE FREE zone. Yet there are four nukes burried in the ice near Thule Air Base ... Greenland ... Danish territory ... Not for sale!
palinka wrote:
2019-12-17 03:55
Absolutely agree. Not making any judgements, just a statement of fact.
Oy vey! So much kvetching! :mrgreen:

Post Reply