SA4 - deadline shrunk

Use this forum for discussions about SpamAssassin and anti-spam in general.
Post Reply
palinka
Senior user
Senior user
Posts: 4593
Joined: 2017-09-12 17:57

SA4 - deadline shrunk

Post by palinka » 2023-10-04 09:50

I'm getting a LOT of these. I use my ISP dns and I signed up for spamhaus DQS. Originally, I thought it was just spamhaus, but its lots of dns queries.

spamd.log:

Code: Select all

Tue Oct 3 16:22:11 2023 [-12472] info: async: aborting after 3.218 s, deadline shrunk: URIBL, A/222.94.231.66.bl.fmb.la, rules: URIBL_FMBLA_A
Tue Oct 3 16:24:57 2023 [-12472] info: async: aborting after 4.041 s, deadline shrunk: DNSBL, TXT/248.52.70.156.sa-accredit.habeas.com, rules: RCVD_IN_VALIDITY_SAFE
Tue Oct 3 16:24:57 2023 [-12472] info: async: aborting after 4.000 s, deadline shrunk: URIBL, A/dns2.p01.nsone.net.nsbl.fmb.la, rules: BODY_NS_URIBL_FMBLA, HEAD_NS_URIBL_FMBLA
Tue Oct 3 16:24:57 2023 [-12472] info: async: aborting after 4.044 s, deadline shrunk: DNSBL, A/248.52.70.156.wl.nszones.com, rules: __RCVD_IN_NSZONES_WL
Tue Oct 3 16:24:57 2023 [-12472] info: async: aborting after 3.989 s, deadline shrunk: URIBL, A/1.45.51.198.bl.fmb.la, rules: URIBL_FMBLA_A
Tue Oct 3 16:45:01 2023 [-12472] info: async: aborting after 3.782 s, deadline shrunk: DNSBL, A/163.12.105.148.psbl.surriel.com, rules: RCVD_IN_PSBL
Tue Oct 3 16:45:01 2023 [-12472] info: async: aborting after 3.783 s, deadline shrunk: DNSBL, A/163.12.105.148.wl.nszones.com, rules: __RCVD_IN_NSZONES_WL
Tue Oct 3 16:45:01 2023 [-12472] info: async: aborting after 3.744 s, deadline shrunk: URIBL, A/65.168.100.95.bl.fmb.la, rules: URIBL_FMBLA_A
Tue Oct 3 16:45:01 2023 [-12472] info: async: aborting after 3.628 s, deadline shrunk: URIBL, A/6.105.74.97.bl.fmb.la, rules: URIBL_FMBLA_A
Tue Oct 3 16:45:01 2023 [-12472] info: async: aborting after 3.742 s, deadline shrunk: URIBL, A/106.32.239.216.bl.fmb.la, rules: URIBL_FMBLA_A
Tue Oct 3 18:28:56 2023 [-12472] info: async: aborting after 3.814 s, deadline shrunk: URIBL, A/b.ns.facebook.com.nsbl.fmb.la, rules: BODY_NS_URIBL_FMBLA, HEAD_NS_URIBL_FMBLA
Tue Oct 3 18:28:56 2023 [-12472] info: async: aborting after 3.832 s, deadline shrunk: DNSBL, A/143.144.220.66.y4ol................yly4.zen.dq.spamhaus.net, rules: __RCVD_IN_ZEN, RCVD_IN_ZEN_BLOCKED, RCVD_IN_XBL, __RCVD_IN_PBL, __RCVD_IN_SBL_CSS, __RCVD_IN_SBL, __RCVD_IN_SBL_DROP, __RCVD_IN_ZEN_LASTEXTERNAL
Is there a setting in SA4 that I missed? My dns seems to be working fine. I did nslookup for about 1/2 the queries above and they all came back as NX Domain. Maybe SA is mistaking NX for timeout? What is happening here?

Of all the queries I tried from the log, none came back as a hit on anything, but I did try some known bad-IP spamhaus queries and they all returned correct values. That leads me to believe SA might be confusing NX for timeout or not seeing the NX Domain result.

palinka
Senior user
Senior user
Posts: 4593
Joined: 2017-09-12 17:57

Re: SA4 - deadline shrunk

Post by palinka » 2023-10-04 18:21

I *think* I found a solution. Its only been a few hours, but the "deadline shrunk" has only hit on AskDNS since implementing, so the logs do appear to have quieted down a lot.

I added this to local.cf, which corresponds to my ISP's DNS servers. It looks like something in SA4 is not finding windows dns. Or there's a setting that I'm just missing. Either way it seems to be working. Need some more time to say this is a definite solution, though.

Code: Select all

dns_server 84.2.44.1
dns_server 84.2.46.1
And WOW! I just received a spam that got absolutely demolished by spamhaus DQS.

Code: Select all

X-Spam-Report: 
 *  0.5 JMQ_SPF_NEUTRAL ASKDNS: SPF set to ?all 
 *      [kudzu.lat TXT:v=spf1 mx ip4:85.93.9.186/32] [?all] 
 * -0.0 SPF_PASS SPF: sender matches SPF record 
 *  0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. 
 *      See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block 
 *      for more information. 
 *      [URI: kudzu.lat] [URI: fonts.googleapis.com] 
 *  8.0 URIBL_DBL_SPAM Contains a spam URL listed in the Spamhaus DBL 
 *      blocklist 
 *      [URI: kudzu.lat] [URI: www.kudzu.lat] 
 *  1.2 URIBL_ABUSE_SURBL Contains an URL listed in the ABUSE SURBL 
 *      blocklist 
 *      [URI: kudzu.lat] [URI: www.kudzu.lat] 
 *  0.1 URIBL_CSS_A Contains URL's A record listed in the Spamhaus CSS 
 *      blocklist 
 *      [URI: kudzu.lat/85.93.9.186] 
 *  6.0 URIBL_ZRD Contains a URL listed in the Spamhaus ZRD blocklist 
 *      [URI: kudzu.lat] 
 *  6.0 SH_ZRD_HEADERS_FRESH A domain found in headers (mail from, reply-to 
 *      etc..) is listed in ZRD and the domain age is between 5 and 24 hours
 *      [kudzu.lat] 
 *  8.0 SH_HELO_DBL The domain used in the HELO string is listed in DBL 
 *      [mail.kudzu.lat] 
 *  0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 
 *  8.0 SH_DBL_HEADERS A domain found in headers (mail from, reply-to etc..)
 *      is listed in DBL 
 *      [kudzu.lat] 
 * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 
 *  0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily 
 *      valid 
 * -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from 
 *      envelope-from domain 
 * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's
 *     domain 
 *  6.0 SH_HELO_ZRD_FRESH The domain used in the HELO string is listed in 
 *      ZRD and the domain age is between 5 and 24 hours 
 *      [mail.kudzu.lat] 
 *  1.5 RCVD_IN_HOSTKARMA_BL RBL: Sender listed in HOSTKARMA-BLACK 
 *      [85.93.9.186 listed in hostkarma.junkemailfilter.com] 
 *  8.0 RCVD_IN_ZEN_LASTEXTERNAL The last untrusted relay is listed in 
 *      Spamhaus ZEN 
 *  1.0 FROM_FMBLA_NEWDOM From domain was registered in last 7 days 
 *  3.0 RCVD_IN_SBL_CSS Received via a relay in Spamhaus SBL-CSS 
 *  5.5 BAYES_999 BODY: Bayes spam probability is 99.9 to 100% 
 *      [score: 1.0000] 
 *  4.5 BAYES_99 BODY: Bayes spam probability is 99 to 100% 
 *      [score: 1.0000] 
 *  0.0 GAMING_SPAM_BODY BODY: Message body contains gaming spam dots and 
 *      underscores 
 *  0.0 HTML_MESSAGE BODY: HTML included in message 
 *  0.0 HTML_IMAGE_ONLY_32 BODY: HTML: images with 2800-3200 bytes of words 
 *  0.0 MIME_QP_LONG_LINE RAW: Quoted-printable line longer than 76 chars 
 *  1.8 CBJ_GiveMeABreak Messages with consecutive break characters 
 *  1.0 KAM_HTMLNOISE Spam containing useless HTML padding 
 *  3.0 KAM_REFI Real Estate / Re-Finance Spam
X-hMailServer-Spam: YES
X-hMailServer-Reason-1: Rejected by Spamhaus. - (Score: 3)
X-hMailServer-Reason-3: Tagged as Spam by SpamAssassin - (Score: 72)
X-hMailServer-Reason-4: Rejected by Spamhaus DBL - (Score: 5)
X-hMailServer-Reason-5: Rejected by SURBL. - (Score: 3)
X-hMailServer-Reason-Score: 83

User avatar
RvdH
Senior user
Senior user
Posts: 3320
Joined: 2008-06-27 14:42
Location: The Netherlands

Re: SA4 - deadline shrunk

Post by RvdH » 2023-10-04 18:40

I have a local BIND9 caching and forwarding server, so in local.cf i have

dns_server [127.0.0.1]:53
dns_server [::1]:53

i still do occasionally get them deadline shrunk messages though randomly, which are basically simple timeouts and nothing to worry about (yours seems to fail on all, that might be troublesome)


PS, note the URIBL_BLOCKED ADMINISTRATOR NOTICE (public DNS / rate limit error)
CIDR to RegEx: d-fault.nl/cidrtoregex
DNS Lookup: d-fault.nl/dnstools
DKIM Generator: d-fault.nl/dkimgenerator
DNSBL Lookup: d-fault.nl/dnsbllookup
GEOIP Lookup: d-fault.nl/geoiplookup

palinka
Senior user
Senior user
Posts: 4593
Joined: 2017-09-12 17:57

Re: SA4 - deadline shrunk

Post by palinka » 2023-10-04 19:31

RvdH wrote:
2023-10-04 18:40
PS, note the URIBL_BLOCKED ADMINISTRATOR NOTICE (public DNS / rate limit error)
That will probably never work for me because I do not plan on installing a DNS server of any kind. My server is win 11 pro. I know there are options, but I simply don't want to deal with it. Maybe in a few years if I run out of things to do.

palinka
Senior user
Senior user
Posts: 4593
Joined: 2017-09-12 17:57

Re: SA4 - deadline shrunk

Post by palinka » 2023-10-10 07:09

palinka wrote:
2023-10-04 18:21
I *think* I found a solution. Its only been a few hours, but the "deadline shrunk" has only hit on AskDNS since implementing, so the logs do appear to have quieted down a lot.

I added this to local.cf, which corresponds to my ISP's DNS servers. It looks like something in SA4 is not finding windows dns. Or there's a setting that I'm just missing. Either way it seems to be working. Need some more time to say this is a definite solution, though.

Code: Select all

dns_server 84.2.44.1
dns_server 84.2.46.1
Update: so far, so good. I'm still getting a few, but I think that's in the normal range now for DNS errors.

Code: Select all

spamd_2023-09-30.log : 162 Results for "deadline shrunk"
spamd_2023-10-01.log : 111 Results for "deadline shrunk"
spamd_2023-10-02.log : 135 Results for "deadline shrunk"
spamd_2023-10-03.log : 135 Results for "deadline shrunk"
spamd_2023-10-04.log : 26 Results for "deadline shrunk"
spamd_2023-10-06.log : 3 Results for "deadline shrunk"
spamd_2023-10-07.log : 8 Results for "deadline shrunk"
spamd_2023-10-08.log : 5 Results for "deadline shrunk"
spamd_2023-10-09.log : 8 Results for "deadline shrunk"
Its weird because I never needed that setting in local.cf in versions 3.x. Maybe it has something to do with my new router. ¯\_(ツ)_/¯

User avatar
RvdH
Senior user
Senior user
Posts: 3320
Joined: 2008-06-27 14:42
Location: The Netherlands

Re: SA4 - deadline shrunk

Post by RvdH » 2023-10-13 09:07

rbl_timeout t [t_min] [zone] (default: 15 3)

All DNS queries are made at the beginning of a check and we try to read the results at the end. This value specifies the maximum period of time (in seconds) to wait for a DNS query. If most of the DNS queries have succeeded for a particular message, then SpamAssassin will not wait for the full period to avoid wasting time on unresponsive server(s), but will shrink the timeout according to a percentage of queries already completed. As the number of queries remaining approaches 0, the timeout value will gradually approach a t_min value, which is an optional second parameter and defaults to 0.2 * t. If t is smaller than t_min, the initial timeout is set to t_min. Here is a chart of queries remaining versus the timeout in seconds, for the default 15 second / 3 second timeout setting:

queries left 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%
timeout 15 14.9 14.5 13.9 13.1 12.0 10.7 9.1 7.3 5.3 3
For example, if 20 queries are made at the beginning of a message check and 16 queries have returned (leaving 20%), the remaining 4 queries should finish within 7.3 seconds since their query started or they will be timed out. Note that timed out queries are only aborted when there is nothing else left for SpamAssassin to do - long evaluation of other rules may grant queries additional time.

If a parameter 'zone' is specified (it must end with a letter, which distinguishes it from other numeric parametrs), then the setting only applies to DNS queries against the specified DNS domain (host, domain or RBL (sub)zone). Matching is case-insensitive, the actual domain may be a subdomain of the specified zone.
The description, especially the zone part isn't very clear if you ask me

But maybe you could try something like the values below to see if it makes a difference

Code: Select all

rbl_timeout 20 5 
20 second timeout, 5 second minimum
CIDR to RegEx: d-fault.nl/cidrtoregex
DNS Lookup: d-fault.nl/dnstools
DKIM Generator: d-fault.nl/dkimgenerator
DNSBL Lookup: d-fault.nl/dnsbllookup
GEOIP Lookup: d-fault.nl/geoiplookup

palinka
Senior user
Senior user
Posts: 4593
Joined: 2017-09-12 17:57

Re: SA4 - deadline shrunk

Post by palinka » 2023-10-13 18:52

RvdH wrote:
2023-10-13 09:07
rbl_timeout t [t_min] [zone] (default: 15 3)

All DNS queries are made at the beginning of a check and we try to read the results at the end. This value specifies the maximum period of time (in seconds) to wait for a DNS query. If most of the DNS queries have succeeded for a particular message, then SpamAssassin will not wait for the full period to avoid wasting time on unresponsive server(s), but will shrink the timeout according to a percentage of queries already completed. As the number of queries remaining approaches 0, the timeout value will gradually approach a t_min value, which is an optional second parameter and defaults to 0.2 * t. If t is smaller than t_min, the initial timeout is set to t_min. Here is a chart of queries remaining versus the timeout in seconds, for the default 15 second / 3 second timeout setting:

queries left 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%
timeout 15 14.9 14.5 13.9 13.1 12.0 10.7 9.1 7.3 5.3 3
For example, if 20 queries are made at the beginning of a message check and 16 queries have returned (leaving 20%), the remaining 4 queries should finish within 7.3 seconds since their query started or they will be timed out. Note that timed out queries are only aborted when there is nothing else left for SpamAssassin to do - long evaluation of other rules may grant queries additional time.

If a parameter 'zone' is specified (it must end with a letter, which distinguishes it from other numeric parametrs), then the setting only applies to DNS queries against the specified DNS domain (host, domain or RBL (sub)zone). Matching is case-insensitive, the actual domain may be a subdomain of the specified zone.
The description, especially the zone part isn't very clear if you ask me

But maybe you could try something like the values below to see if it makes a difference

Code: Select all

rbl_timeout 20 5 
20 second timeout, 5 second minimum
How often do DNS queries take more than 1 second to complete? If it gets to 14 seconds - the longest "deadline shrunk" that I've seen in my logs - then I'll assume no amount of time is going to return a result. Even 3 seconds is pretty dubious. Extending the timeout looks like it will just extend the time required to process a message - but getting the same results.

By the way, is this something new to SA4? I never saw this until I upgraded.

User avatar
johang
Senior user
Senior user
Posts: 1154
Joined: 2008-09-01 09:20

Re: SA4 - deadline shrunk

Post by johang » 2023-10-15 12:02

it could differ/depend if you are letting SA do the query through a stub resolver or an itterative resolver

when using SA its "always" best to run yur own itterative resolver, since you have no control of an external one (you do not know why it might timeout.. )

i would say 10 seconds om your stub resolver and 5 on an itterative resolver.. perhaps you could tweak it to 2 seconds if you have control of your own itterative resolver ( for instance running Bind on your server which is set to doing direct dns resolving and not in its turn use any other resolver like opendns or google )
lets cheat darwin out of his legacy, find a cure for cancer...

User avatar
RvdH
Senior user
Senior user
Posts: 3320
Joined: 2008-06-27 14:42
Location: The Netherlands

Re: SA4 - deadline shrunk

Post by RvdH » 2024-05-29 09:52

palinka wrote:
2023-10-04 09:50

Code: Select all

Tue Oct 3 16:24:57 2023 [-12472] info: async: aborting after 4.044 s, deadline shrunk: DNSBL, A/248.52.70.156.wl.nszones.com, rules: __RCVD_IN_NSZONES_WL
You probably should ditch the nsZones.com, which has been proven fake and was in the news negatively years ago
https://web.archive.org/web/20230514155 ... szones.com
Fraudulent fake DNSBL uncovered: nszones.com
2009-12-10. Spamhaus has uncovered a fake spam blocklist which was pirating and selling as it's own work DNSBL data stolen from major anti-spam systems including Spamhaus, CBL and SURBL, republishing the stolen data under the name "nszones.com".

Nszones operates a 'remove your IP' scam charging unsuspecting internet users a fee to be 'removed' from the pirated nszones DNSBLs. The operation also attempts to sell 'commercial subscriptions' to nszones' pirated DNSBLs.

Owned by Liberian-registered Aegeas Enterprises S.A. based in Greece, nszones.com was discovered pirating Spamhaus DNSBL data via an rsync service Spamhaus provided for some customers and 3rd party data services, and was found to be republishing the pirated data under the hostnames 'bl.nszones.com, sbl.nszones.com, dyn.nszones.com and ubl.nszones.com'. Secret seed data which Spamhaus adds into Spamhaus DNSBL zones to catch data pirates was found in bl.nszones.com, sbl.nszones.com and dyn.nszones.com.

Similarly, nszones' whitelist wl.nszones.com was found to consist of data verified to have been wholly pirated from whitelist service DNSWL.org.

Aegeas Enterprises S.A. which operates as aegeas.com, spamfilter.corp.gr, corp.gr, 1net.gr, hostingkey.gr, nskey.com and nszones.com, was blocked from accessing the Spamhaus and CBL rsync servers in December 2009 and was sent a formal Cease & Desist by Spamhaus. However as the nszones operation is both anonymous and likely makes some -albeit probably small- profit from the fake delisting fee scam, the actor behind nszones may simply leave the site up while still linked to by any unsuspecting 'Check your IP' sites.
CIDR to RegEx: d-fault.nl/cidrtoregex
DNS Lookup: d-fault.nl/dnstools
DKIM Generator: d-fault.nl/dkimgenerator
DNSBL Lookup: d-fault.nl/dnsbllookup
GEOIP Lookup: d-fault.nl/geoiplookup

User avatar
RvdH
Senior user
Senior user
Posts: 3320
Joined: 2008-06-27 14:42
Location: The Netherlands

Re: SA4 - deadline shrunk

Post by RvdH » 2024-05-29 09:57

FYI, another fake one discovered/identified by Spamhaus people

https://web.archive.org/web/20230414160 ... ad.psky.me
Fraudulent fake DNSBL uncovered: Protected Sky (bad.psky.me)
2017-03-27. Spamhaus has uncovered an anonymously-run DNSBL service which was wholly pirating our data and republishing it as its own work. "Protected Sky" (PSKY) published a DNSBL under the name 'bad.psky.me'. The Protected Sky web site contained almost no information on its data, listing policies or delisting procedures, nor offered any way to request delisting/removal of any listed IP (one of the most important DNSBL Best Practice requirements as defined by RFC6471).

Suspicions were raised when Spamhaus noticed that IP addresses recently listed on Spamhaus DNSBLs would consistently appear in bad.psky.me a short time later. Conversely, IP addresses removed from Spamhaus DNSBLs would disappear from bad.psky.me a short time later. Spamhaus listings made using specially developed methodologies and intelligence and which were thus not likely to be detected or listed by other DNSBLs were found consistently republished in bad.psky.me. On closer inspection no Spamhaus listing was found to not be republished a consistently short time later in bad.psky.me.

3rd parties also observed close synchronization between Spamhaus listings and removals and PSKY listings and removals (ref: Webhostingtalk, Webhostingtalk, Hetrixtools, SpamAssassin mailing list). However, user comments (as seen in the links above) also indicated high false positive rates suggesting that Protected Sky was also pirating data from others including from not-too-reliable or stale data sources in addition to the data it was pirating from Spamhaus. Spamhaus therefore suggested that other DNSBL operators check whether data from their DNSBLs may have also been poached by PSKY.

Protected Sky's access to Spamhaus DNSBL data was traced back through an unaware third party customer of the Spamhaus Datafeed service. As of 23 March 2017 additional security measures were put in place by the customer and Spamhaus to block further unauthorized access to the Datafeed service.


In 2009 Spamhaus exposed a similarly fraudulent fake DNSBL; nsZones.com, which, like Protected Sky, was simply stealing and republishing Spamhaus data as its own work for the purpose of charging unsuspecting users fees to pretend to remove IPs from nsZones.com.
CIDR to RegEx: d-fault.nl/cidrtoregex
DNS Lookup: d-fault.nl/dnstools
DKIM Generator: d-fault.nl/dkimgenerator
DNSBL Lookup: d-fault.nl/dnsbllookup
GEOIP Lookup: d-fault.nl/geoiplookup

palinka
Senior user
Senior user
Posts: 4593
Joined: 2017-09-12 17:57

Re: SA4 - deadline shrunk

Post by palinka » 2024-05-29 17:05

Good to know. Thanks.

Post Reply