SpamAssassin Custom Rules

Use this forum for discussions about SpamAssassin and anti-spam in general.
User avatar
RvdH
Senior user
Senior user
Posts: 608
Joined: 2008-06-27 14:42
Location: Netherlands

SpamAssassin Custom Rules

Post by RvdH » 2015-07-18 11:23

I have found some custom SpamAssassin Rules, that should trigger on phishing attempts by Dutch banks, https://github.com/hspaans/spamassassin-local-rules (80_phishing.cf)

Code: Select all

# Domains:
# abnamro.nl 
# nl.abnamro.com
describe PHISH_FROM_ABNAMRO        Trigger on phishing mails
header   PHISH_FROM_ABNAMRO        From:addr =~ /\@(abnamro\.nl|nl\.abnamro\.com)$/i
score    PHISH_FROM_ABNAMRO        2.0

describe UNPHISH_FROM_ABNAMRO      Untrigger on valid mails
header   __UNPHISH_FROM_ABNAMRO_A  Return-Path:addr =~ /\@(abnamro\.nl|nl\.abnamro\.com)$/i
meta     UNPHISH_FROM_ABNAMRO      ( __UNPHISH_FROM_ABNAMRO_A && SPF_PASS )
score    UNPHISH_FROM_ABNAMRO      -2.0
The first rule should trigger on anything with a FROM header of @abnambro.nl and @nl.abnamro.com and add to the score
The second rule should trigger on SPF_PASS and the Return-Path header contains header of @abnambro.nl and @nl.abnamro.com and lower the score

Funny thing is, in my spamd.exe logfile i have absolutely no hits on SPF_PASS, the only thing that comes near is SPF_HELO_PASS
Anyone can tell me why SPF_PASS is not triggered?
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
jimimaseye
Moderator
Moderator
Posts: 7624
Joined: 2011-09-08 17:48

Re: SpamAssassin Custom Rules

Post by jimimaseye » 2015-07-18 13:20

Check its included in the SPF list of checks.

C:\Program Files (x86)\JAM Software\SpamAssassin for Windows\share\3.004000\updates_spamassassin_org\25_SPF.cf
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
RvdH
Senior user
Senior user
Posts: 608
Joined: 2008-06-27 14:42
Location: Netherlands

Re: SpamAssassin Custom Rules

Post by RvdH » 2015-07-18 16:33

Yep, it is included in 25_SPF.cf

If i search my logs the only hits i find are:

SPF_HELO_PASS (787 matches)
SPF_HELO_NEUTRAL (4 matches)
SPF_HELO_FAIL (32 matches)
SPF_HELO_SOFTFAIL (10 matches)

None of the 'normal' SPF check have any hits

SPF_PASS (0 matches)
SPF_NEUTRAL (0 matches)
SPF_FAIL (0 matches)
SPF_SOFTFAIL (0 matches)

Weird, very weird :?:
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
jimimaseye
Moderator
Moderator
Posts: 7624
Joined: 2011-09-08 17:48

Re: SpamAssassin Custom Rules

Post by jimimaseye » 2015-07-18 17:31

Dunno then. Ive got many of all of them.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
SorenR
Senior user
Senior user
Posts: 2690
Joined: 2006-08-21 15:38
Location: Denmark

Re: SpamAssassin Custom Rules

Post by SorenR » 2015-07-18 18:16

Since Feb. 12...

SPF_PASS (7 matches)
SPF_NEUTRAL (0 matches)
SPF_FAIL (0 matches)
SPF_SOFTFAIL (0 matches)

Who is using the free edition of "Jam-Ware" and who is using "Box" version with additional rules etc. ?
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
RvdH
Senior user
Senior user
Posts: 608
Joined: 2008-06-27 14:42
Location: Netherlands

Re: SpamAssassin Custom Rules

Post by RvdH » 2015-07-18 18:47

Code: Select all

Sat Jul 18 18:41:49 2015 [-7680] info: spamd: connection from Server [127.0.0.1]:60004 to port 783, fd 6
Sat Jul 18 18:41:49 2015 [-7680] info: spamd: processing message <014305680150.87565368802183@gewix.com> for (unknown):0
Sat Jul 18 18:41:50 2015 [-7680] dbg: spf: checking to see if the message has a Received-SPF header that we can use
Sat Jul 18 18:41:50 2015 [-7680] dbg: spf: checking HELO (helo=191.pichincha.andinanet.net, ip=190.152.41.191)
Sat Jul 18 18:41:50 2015 [-7680] dbg: spf: query for /190.152.41.191/191.pichincha.andinanet.net: result: none, comment: , text: No applicable sender policy available
Sat Jul 18 18:41:50 2015 [-7680] dbg: spf: already checked for Received-SPF headers, proceeding with DNS based checks
Sat Jul 18 18:41:50 2015 [-7680] dbg: spf: cannot get Envelope-From, cannot use SPF
Sat Jul 18 18:41:50 2015 [-7680] dbg: spf: def_spf_whitelist_from: already checked spf and didn't get pass, skipping whitelist check
Sat Jul 18 18:41:50 2015 [-7680] dbg: spf: whitelist_from_spf: already checked spf and didn't get pass, skipping whitelist check
Sat Jul 18 18:41:50 2015 [-7680] info: spamd: identified spam (20.8/4.0) for (unknown):0 in 1.0 seconds, 1402 bytes.
Sat Jul 18 18:41:50 2015 [-7680] info: spamd: result: Y 20 - BAYES_99,BAYES_999,CUSTOM_MANY_URIBL,LOCAL_SCAM_8,RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK,RCVD_IN_BRBL_LASTEXT,RCVD_IN_NIX_SPAM,URIBL_BLACK,URIBL_JP_SURBL,URIBL_MW_SURBL scantime=1.0,size=1402,user=(unknown),uid=0,required_score=4.0,rhost=Server,raddr=127.0.0.1,rport=60004,mid=<014305680150.87565368802183@gewix.com>,bayes=1.000000,autolearn=no autolearn_force=no,shortcircuit=no
This one looks fishy, don't you think?
Sat Jul 18 18:41:50 2015 [-7680] dbg: spf: cannot get Envelope-From, cannot use SPF
envelope_sender_header Name-Of-Header
SpamAssassin will attempt to discover the address used in the 'MAIL FROM:' phase of the SMTP transaction that delivered this message, if this data has been made available by the SMTP server. This is used in the EnvelopeFrom pseudo-header, and for various rules such as SPF checking.

By default, various MTAs will use different headers, such as the following:

X-Envelope-From
Envelope-Sender
X-Sender
Return-Path
SpamAssassin will attempt to use these, if some heuristics (such as the header placement in the message, or the absence of fetchmail signatures) appear to indicate that they are safe to use. However, it may choose the wrong headers in some mailserver configurations. (More discussion of this can be found in bug 2142 and bug 4747 in the SpamAssassin BugZilla.)

To avoid this heuristic failure, the envelope_sender_header setting may be helpful. Name the header that your MTA or MDA adds to messages containing the address used at the MAIL FROM step of the SMTP transaction.

If the header in question contains < or > characters at the start and end of the email address in the right-hand side, as in the SMTP transaction, these will be stripped.

If the header is not found in a message, or if it's value does not contain an @ sign, SpamAssassin will issue a warning in the logs and fall back to its default heuristics.

(Note for MTA developers: we would prefer if the use of a single header be avoided in future, since that precludes 'downstream' spam scanning. http://wiki.apache.org/spamassassin/Env ... InReceived details a better proposal, storing the envelope sender at each hop in the Received header.)

example:

envelope_sender_header X-SA-Exim-Mail-From
Do you guys have "envelope_sender_header" set in local.cf?


@SorenR, i am using the Free Edition, with the body_0 compiled rules and jam.cf and jam_virus_bounce_rules.cf from the BOX demo/trial version
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
jimimaseye
Moderator
Moderator
Posts: 7624
Joined: 2011-09-08 17:48

Re: SpamAssassin Custom Rules

Post by jimimaseye » 2015-07-18 19:12

SorenR wrote:Who is using the free edition of "Jam-Ware" and who is using "Box" version with additional rules etc. ?
I am using bog standard free edition.
RvdH wrote: Do you guys have "envelope_sender_header" set in local.cf?
No. My local.cf is detailed in my 'Installation' tutorial.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
jimimaseye
Moderator
Moderator
Posts: 7624
Joined: 2011-09-08 17:48

Re: SpamAssassin Custom Rules

Post by jimimaseye » 2015-07-18 19:20

Are you using direct SMTP delivery or External Download collect for you email?

Ive just done a little test: all email that is collected by External Download DOES have the SPF_PASS, yet the (2 test examples) I sent in using direct SMTP do not.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
RvdH
Senior user
Senior user
Posts: 608
Joined: 2008-06-27 14:42
Location: Netherlands

Re: SpamAssassin Custom Rules

Post by RvdH » 2015-07-18 19:21

I added to local.cf:

envelope_sender_header Return-Path

Code: Select all

Sat Jul 18 19:14:34 2015 [-7300] info: spamd: processing message <03dc0c45c95b4f4311ad10736020587b089a0e1d@mails.watisdebeste.nl> for (unknown):0
Sat Jul 18 19:14:34 2015 [-7300] dbg: spf: checking to see if the message has a Received-SPF header that we can use
Sat Jul 18 19:14:34 2015 [-7300] dbg: spf: checking HELO (helo=mail-195-191-144-46.simpelpmta1-3.com, ip=195.191.144.46)
Sat Jul 18 19:14:34 2015 [-7300] dbg: spf: query for /195.191.144.46/mail-195-191-144-46.simpelpmta1-3.com: result: none, comment: , text: No applicable sender policy available
Sat Jul 18 19:14:34 2015 [-7300] dbg: spf: already checked for Received-SPF headers, proceeding with DNS based checks
Sat Jul 18 19:14:34 2015 [-7300] dbg: spf: checking EnvelopeFrom (helo=mail-195-191-144-46.simpelpmta1-3.com, ip=195.191.144.46, envfrom=22414-710514445-0edfc4aff8@bounce.mails.watisdebeste.nl)
Sat Jul 18 19:14:34 2015 [-7300] dbg: spf: query for 22414-710514445-0edfc4aff8@bounce.mails.watisdebeste.nl/195.191.144.46/mail-195-191-144-46.simpelpmta1-3.com: result: pass, comment: , text: Mechanism 'ip4:195.191.144.0/24' matched
Sat Jul 18 19:14:34 2015 [-7300] dbg: spf: def_whitelist_from_spf: 22414-710514445-0edfc4aff8@bounce.mails.watisdebeste.nl is not in DEF_WHITELIST_FROM_SPF
Sat Jul 18 19:14:34 2015 [-7300] dbg: spf: whitelist_from_spf: 22414-710514445-0edfc4aff8@bounce.mails.watisdebeste.nl is not in user's WHITELIST_FROM_SPF
Sat Jul 18 19:14:36 2015 [-7300] info: spamd: clean message (0.9/4.0) for (unknown):0 in 1.7 seconds, 14885 bytes.
Sat Jul 18 19:14:36 2015 [-7300] info: spamd: result: . 0 - BAYES_20,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HTML_MESSAGE,RDNS_DYNAMIC,SPF_PASS scantime=1.7,size=14885,user=(unknown),uid=0,required_score=4.0,rhost=Server,raddr=127.0.0.1,rport=60206,mid=<03dc0c45c95b4f4311ad10736020587b089a0e1d@mails.watisdebeste.nl>,bayes=0.197003,autolearn=no autolearn_force=no,shortcircuit=no

Code: Select all

Sat Jul 18 19:15:17 2015 [-7300] info: spamd: processing message <000001d0c17d$47256450$d5702cf0$@guusenpetra.nl> for (unknown):0
Sat Jul 18 19:15:17 2015 [-7300] dbg: spf: checking to see if the message has a Received-SPF header that we can use
Sat Jul 18 19:15:17 2015 [-7300] dbg: spf: checking HELO (helo=emea01-db3-obe.outbound.protection.outlook.com, ip=157.55.234.123)
Sat Jul 18 19:15:17 2015 [-7300] dbg: spf: query for /157.55.234.123/emea01-db3-obe.outbound.protection.outlook.com: result: pass, comment: , text: Mechanism 'include:spf.protection.outlook.com' matched
Sat Jul 18 19:15:17 2015 [-7300] dbg: spf: already checked for Received-SPF headers, proceeding with DNS based checks
Sat Jul 18 19:15:17 2015 [-7300] dbg: spf: checking EnvelopeFrom (helo=emea01-db3-obe.outbound.protection.outlook.com, ip=157.55.234.123, envfrom=admin@guusenpetra.nl)
Sat Jul 18 19:15:17 2015 [-7300] dbg: spf: query for admin@guusenpetra.nl/157.55.234.123/emea01-db3-obe.outbound.protection.outlook.com: result: pass, comment: , text: Mechanism 'include:spf.protection.outlook.com' matched
Sat Jul 18 19:15:17 2015 [-7300] dbg: spf: def_whitelist_from_spf: admin@guusenpetra.nl is not in DEF_WHITELIST_FROM_SPF
Sat Jul 18 19:15:17 2015 [-7300] dbg: spf: whitelist_from_spf: admin@guusenpetra.nl is not in user's WHITELIST_FROM_SPF
Sat Jul 18 19:15:19 2015 [-7300] info: spamd: clean message (-0.9/4.0) for (unknown):0 in 2.1 seconds, 48919 bytes.
Sat Jul 18 19:15:19 2015 [-7300] info: spamd: result: . 0 - BAYES_00,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,RCVD_IN_JUSTSPAM,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS scantime=2.1,size=48919,user=(unknown),uid=0,required_score=4.0,rhost=Server,raddr=127.0.0.1,rport=60223,mid=<000001d0c17d$47256450$d5702cf0$@guusenpetra.nl>,bayes=0.000577,autolearn=no autolearn_force=no,shortcircuit=no
Now i have my first SPF_PASS entries :wink:

[Edit]
No External Download of e-mail on this server
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
SorenR
Senior user
Senior user
Posts: 2690
Joined: 2006-08-21 15:38
Location: Denmark

Re: SpamAssassin Custom Rules

Post by SorenR » 2015-07-18 19:53

Hmm... I did some tests yesterday and also intercepted the returnfile from SpamAssassin...

NO "Return-Path: sender@domain.com" to be found anywhere in the email... Weird as it IS there when email is delivered into INBOX... :?

There is an actual test to see if Envelope sender, From: and Return-Path: match.... 20_head_tests.cf and 20_vbounce.cf ...

This is the actual code I pasted into every relevant trigger in EventHandlers.vbs

Code: Select all

   On Error Resume Next
      EventLog.Write("OnSMTPData(oClient, oMessage) **********************************************")
      EventLog.Write("MAIL FROM: " & oMessage.FromAddress)
      Dim i, oFSO, oTS
      For i = 0 To oMessage.Headers.Count-1
         EventLog.Write(oMessage.Headers(i).Name & ": " & oMessage.Headers(i).Value)
      Next
      EventLog.Write("FILEDATA *******************************************************************")
      Set oFSO = CreateObject("Scripting.FileSystemObject")
      Set oTS = oFSO.OpenTextFile(oMessage.Filename)
      Do Until oTS.AtEndOfStream
         EventLog.Write(oTS.ReadLine)
      Loop
      Set oTS = Nothing
      Set oFSO = Nothing
      EventLog.Write("****************************************************************************")
   On Error Goto 0
Error handling is simply to NOT disturb existing code :mrgreen:
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
RvdH
Senior user
Senior user
Posts: 608
Joined: 2008-06-27 14:42
Location: Netherlands

Re: SpamAssassin Custom Rules

Post by RvdH » 2015-07-18 19:56

Question still remains why SpamAssassin is not able to find the EnvelopeFrom pseudo-header by it's own... The only thing i can think of right now is fact i enabled the Delivered-To header in hMailServer

gonna test some more...
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
SorenR
Senior user
Senior user
Posts: 2690
Joined: 2006-08-21 15:38
Location: Denmark

Re: SpamAssassin Custom Rules

Post by SorenR » 2015-07-18 19:58

FYI... "envelope_sender_header Name-Of-Header" about half way down the page...

http://spamassassin.apache.org/full/3.4 ... _Conf.html
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
SorenR
Senior user
Senior user
Posts: 2690
Joined: 2006-08-21 15:38
Location: Denmark

Re: SpamAssassin Custom Rules

Post by SorenR » 2015-07-18 19:59

RvdH wrote:Question still remains why SpamAssassin is not able to find the EnvelopeFrom pseudo-header by it's own... The only thing i can think of right now is fact i enabled the Delivered-To header in hMailServer

gonna test some more...
It's not there, that's why... :mrgreen:

"Delivered-To" header suffers from exactly the same problem. It magicly appears just before the email is dumped into users IMAP folder...
Last edited by SorenR on 2015-07-18 20:01, edited 1 time in total.
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
jimimaseye
Moderator
Moderator
Posts: 7624
Joined: 2011-09-08 17:48

Re: SpamAssassin Custom Rules

Post by jimimaseye » 2015-07-18 19:59

SorenR wrote:Hmm... I did some tests yesterday and also intercepted the returnfile from SpamAssassin...

NO "Return-Path: sender@domain.com" to be found anywhere in the email... Weird as it IS there when email is delivered into INBOX... :?
Thats an interesting find. And actually goes some to explain my test results: even WITH the 'envelope_sender_header Return-Path' parameter added to local.cf (as OP said he did and got a result), it still didnt make a difference for me (and yet the 'return-path' header is definitely in evidence in the message when you view it). If the header is available to SA (as you say) then it isnt going to use it.

So, with or without "envelope_sender_header Return-Path" I never got the SPF_PASS check direct SMTP deliveries and yet on an External Download it is always there (because I assume SA is able to read the 'return path' header as it will already have been completed and filled out).
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
RvdH
Senior user
Senior user
Posts: 608
Joined: 2008-06-27 14:42
Location: Netherlands

Re: SpamAssassin Custom Rules

Post by RvdH » 2015-07-18 20:21

if you add "--debug spf" to spamd.exe parameters, you can see all debug info for spf related checks
SorenR wrote:FYI... "envelope_sender_header Name-Of-Header" about half way down the page...

http://spamassassin.apache.org/full/3.4 ... _Conf.html

i know, i already quoted it here
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
SorenR
Senior user
Senior user
Posts: 2690
Joined: 2006-08-21 15:38
Location: Denmark

Re: SpamAssassin Custom Rules

Post by SorenR » 2015-07-18 20:31

jimimaseye wrote:
SorenR wrote:Hmm... I did some tests yesterday and also intercepted the returnfile from SpamAssassin...

NO "Return-Path: sender@domain.com" to be found anywhere in the email... Weird as it IS there when email is delivered into INBOX... :?
Thats an interesting find. And actually goes some to explain my test results: even WITH the 'envelope_sender_header Return-Path' parameter added to local.cf (as OP said he did and got a result), it still didnt make a difference for me (and yet the 'return-path' header is definitely in evidence in the message when you view it). If the header is available to SA (as you say) then it isnt going to use it.

So, with or without "envelope_sender_header Return-Path" I never got the SPF_PASS check direct SMTP deliveries and yet on an External Download it is always there (because I assume SA is able to read the 'return path' header as it will already have been completed and filled out).
Presumable...

I tried to add it to the headers under the OnSMTPData trigger with oMessage.Headervalue("Return-Path") = oMessage.FromAddress and oMessage.Save... Made a complete FU on all the headers :oops:
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
RvdH
Senior user
Senior user
Posts: 608
Joined: 2008-06-27 14:42
Location: Netherlands

Re: SpamAssassin Custom Rules

Post by RvdH » 2015-07-18 21:55

You can also set it like:

Code: Select all

envelope_sender_header From 
This enables the SPF checks as well, if you like to test



X-Envelope-From
Envelope-Sender
X-Sender
Return-Path

SpamAssassin will attempt to use these, if some heuristics (such as the header
placement in the message, or the absence of fetchmail signatures) appear to
indicate that they are safe to use. However, it may choose the wrong headers
in some mailserver configurations. (More discussion of this can be found
in bug 2142 and bug 4747 in the SpamAssassin BugZilla.)
So basically i think we found a bug in hmailserver, the MTA (hmailserver in this case) has to add one off the above headers to the message before it sends it to SpamAssasin
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
RvdH
Senior user
Senior user
Posts: 608
Joined: 2008-06-27 14:42
Location: Netherlands

Re: SpamAssassin Custom Rules

Post by RvdH » 2015-07-18 22:53

SorenR wrote: I tried to add it to the headers under the OnSMTPData trigger with oMessage.Headervalue("Return-Path") = oMessage.FromAddress and oMessage.Save... Made a complete FU on all the headers :oops:
I would say that if you make hMailServer add headers like X-hMailServer-Spam, X-hMailServer-Reason-Score the SA reports are always FU, hMailServer destroys the SA reports markup
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
jimimaseye
Moderator
Moderator
Posts: 7624
Joined: 2011-09-08 17:48

Re: SpamAssassin Custom Rules

Post by jimimaseye » 2015-07-19 10:29

RvdH wrote:I would say that if you make hMailServer add headers like X-hMailServer-Spam, X-hMailServer-Reason-Score the SA reports are always FU, hMailServer destroys the SA reports markup
What do you mean by this?

FYI a typical copy of my headers (when tagged by hmailserver as spam (rare!)) looks like this:

Code: Select all

Return-Path: geoff@crow.co.uk
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mailserver
X-Spam-Flag: YES
X-Spam-Level: *****
X-Spam-Status: Yes, score=5.1 required=3.0 tests=BAYES_00,HTML_MESSAGE, MY_PHISH_BODY
 autolearn=no autolearn_force=no version=3.4.0
X-Spam-Report: 
 * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%
 *      [score: 0.0000]
 *  0.0 HTML_MESSAGE BODY: HTML included in message
 *  7.0 MY_PHISH_BODY pay/itun/App/acc (verif|update/restore/validate/unlock etc)
 *
Return-Path: <geoff@crow.co.uk>
Received: from mailin3.host.com (mailin3.host.com [196.2.90.13]) (authenticated
 user=me@mycompany.com bits=0) by ms7.host.com (Cyrus v2.4.16-Kolab-2.4.16-1.el6)
 with LMTPSA (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256/256
 verify=YES); Thu, 26 Mar 2015 15:44:55 +0000
X-Sieve: CMU Sieve 2.4
Received: from 14.182-104-20.static.mediabusiness.co.uk ([21.114.182.200] helo=remote.crow.co.uk)
 by mailin.host.com with esmtps (TLSv1:AES128-SHA:128) (Exim
 4.82_1-5b7a7c0-XX) (envelope-from <geoff@crow.co.uk>) id
 1Yb9xl-0002sU-HW; Thu, 26 Mar 2015 15:44:55 +0000
Received: from SPCSBS.spc.local ([fe80::4450:fa46:224b:c6fa]) by SPCSBS.spc.local ([fe80::4450:fa46:224b:c6fa%10])
 with mapi id 14.02.0387.000; Thu, 26 Mar 2015 15:44:50 +0000
From: geoff rey <geoff@crow.co.uk>
To: geoff rey <geoff@crow.co.uk>
Subject: [SPAM] [5.1] P11Ds for the tax year 2014/15
Date: Thu, 26 Mar 2015 15:44:49 +0000
X-Spam-Prev-Subject: P11Ds for the tax year 2014/15
X-hMailServer-Spam: YES
X-hMailServer-Reason-1: Tagged as Spam by SpamAssassin - (Score: 5)
X-hMailServer-Reason-Score: 5
What do you see as wrong with that?
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
RvdH
Senior user
Senior user
Posts: 608
Joined: 2008-06-27 14:42
Location: Netherlands

Re: SpamAssassin Custom Rules

Post by RvdH » 2015-07-19 10:39

Code: Select all

X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) * on Server * at Sun, 19
 Jul 2015 08:13:19 +0200
X-Spam-Status: Yes, score=5.4, hits=5.4, required=4.0, autolearn=no autolearn_force=no,
 shortcircuit=no
X-Spam-Level: *****
X-Spam-Report: *  0.7 SPF_SOFTFAIL SPF: sender does not match SPF record (softfail) *  1.5
 SUBJ_ALL_CAPS Subject is all capitals *  0.2 JAM_LONG_LINK BODY: Very long
 link in mail, possibly filled up with *      random words by bulk mailer *
 0.4 HTML_IMAGE_RATIO_02 BODY: HTML has a low ratio of text to image area *
  0.0 HTML_MESSAGE BODY: HTML included in message *  1.0 HTML_FONT_FACE_BAD
 BODY: HTML font face is not a word *  0.8 BAYES_50 BODY: Bayes spam
 probability is 40 to 60% *      [score: 0.4965] *  0.8 MPART_ALT_DIFF
 BODY: HTML and text parts are different
...
X-hMailServer-Spam: YES
X-hMailServer-Reason-1: Tagged as Spam by SpamAssassin - (Score: 5)
X-hMailServer-Reason-Score: 5
As you can see the X-Spam-Report header markup is gone, looks like line breaks are missing or something

Without the X-hMailServer headers, it looks ok, eg:

Code: Select all

X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) * on
	Server * at Sun, 19 Jul 2015 08:13:19 +0200
X-Spam-Status: No, score=-7.1, hits=-7.1, required=4.0, autolearn=no
	autolearn_force=no, shortcircuit=no
X-Spam-Report: 
	* -6.0 USER_IN_WHITELIST_TO User is listed in 'whitelist_to'
	* -0.0 SPF_PASS SPF: sender matches SPF record
	*  0.8 PDF_ATTACHED BODY: email contains a PDF file attachment
	* -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%
	*      [score: 0.0000]
Do you use hMailServer or SpamAssassin to alter the message subject? (eg: prefix it with [SPAM])

I notice a difference with my result, eg:
Subject: [SPAM] [5.1] P11Ds for the tax year 2014/15
X-Spam-Prev-Subject: P11Ds for the tax year 2014/15
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
jimimaseye
Moderator
Moderator
Posts: 7624
Joined: 2011-09-08 17:48

Re: SpamAssassin Custom Rules

Post by jimimaseye » 2015-07-19 12:03

Ah yes, I see what you mean. And in fact I have just checked and it is the same for me too (the addition of HMS headers does break up the SA Report header format as you have pointed out. The example I gave you I had 'reformatted' before posting). Its not ideal, no, but its not a show stopper for me personally (as I say I rarely get HMS adding [spam] to genuine emails - I have had only 8 this year out of 2544 emails on one account - and even then its not so important to read the SA headers for me).
RvdH wrote: Do you use hMailServer or SpamAssassin to alter the message subject? (eg: prefix it with [SPAM])

I notice a difference with my result, eg:
Subject: [SPAM] [5.1] P11Ds for the tax year 2014/15
X-Spam-Prev-Subject: P11Ds for the tax year 2014/15
SA adds the score to my subject if it exceeds the threshold (ie, the "[5.1]"). It is HMS that adds the "[SPAM]" part afterwards.

(Regarding the "X-Spam-Prev-Subject", I cant say I had noticed that before (Never really looked). Ooohh...)

My relevant config settings:

Code: Select all

report_safe 0
add_header all Report _REPORT_ *
rewrite_header Subject [_HITS_]
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
RvdH
Senior user
Senior user
Posts: 608
Joined: 2008-06-27 14:42
Location: Netherlands

Re: SpamAssassin Custom Rules

Post by RvdH » 2015-07-19 13:50

I opened a ticket (#115) about the header reformatting by hMailServer


But back to the subject,.
do you guys think it is a bug hMailServer that is doesn't supply the EnvelopeFrom pseudo-header (required for SPF checks) SA expects?
envelope_sender_header Name-Of-Header
SpamAssassin will attempt to discover the address used in the 'MAIL FROM:' phase of the SMTP transaction that delivered this message, if this data has been made available by the SMTP server. This is used in the EnvelopeFrom pseudo-header, and for various rules such as SPF checking.

By default, various MTAs will use different headers, such as the following:

X-Envelope-From
Envelope-Sender
X-Sender
Return-Path
SpamAssassin will attempt to use these, if some heuristics (such as the header placement in the message, or the absence of fetchmail signatures) appear to indicate that they are safe to use. However, it may choose the wrong headers in some mailserver configurations. (More discussion of this can be found in bug 2142 and bug 4747 in the SpamAssassin BugZilla.)

To avoid this heuristic failure, the envelope_sender_header setting may be helpful. Name the header that your MTA or MDA adds to messages containing the address used at the MAIL FROM step of the SMTP transaction.

If the header in question contains < or > characters at the start and end of the email address in the right-hand side, as in the SMTP transaction, these will be stripped.

If the header is not found in a message, or if it's value does not contain an @ sign, SpamAssassin will issue a warning in the logs and fall back to its default heuristics.

(Note for MTA developers: we would prefer if the use of a single header be avoided in future, since that precludes 'downstream' spam scanning. http://wiki.apache.org/spamassassin/Env ... InReceived details a better proposal, storing the envelope sender at each hop in the Received header.)

example:

envelope_sender_header X-SA-Exim-Mail-From

Maybe adding a header like: X-hMailServer-Envelope-From before sending the message to SA can fix the issue "cannot get Envelope-From, cannot use SPF"

local.cf
envelope_sender_header X-hMailServer-Envelope-From
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
jimimaseye
Moderator
Moderator
Posts: 7624
Joined: 2011-09-08 17:48

Re: SpamAssassin Custom Rules

Post by jimimaseye » 2015-07-19 16:27

RvdH wrote: do you guys think it is a bug hMailServer that is doesn't supply the EnvelopeFrom pseudo-header (required for SPF checks) SA expects?
I woudnt call it a bug as their write up clearly suggests that it is expected that some MTAs dont do it:
SpamAssassin will attempt to discover the address used in the 'MAIL FROM:' phase of the SMTP transaction that delivered this message, if this data has been made available by the SMTP server.
However, it is strange that the RETURN-PATH isnt available until the final delivery to HMS (even after it has been through SA) as Soren discovered above. Maybe this is a bug.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
RvdH
Senior user
Senior user
Posts: 608
Joined: 2008-06-27 14:42
Location: Netherlands

Re: SpamAssassin Custom Rules

Post by RvdH » 2015-07-19 16:37

No, that is by design i think, see:
RFC2821:
When the delivery SMTP server makes the "final delivery" of a
message, it inserts a return-path line at the beginning of the mail
data. This use of return-path is required; mail systems MUST support
it. The return-path line preserves the information in the <reverse-
path> from the MAIL command. Here, final delivery means the message
has left the SMTP environment. Normally, this would mean it had been
delivered to the destination user or an associated mail drop, but in
some cases it may be further processed and transmitted by another
mail system.
I think the MTA (hMailServer) has to provide one of these: X-Envelope-From, Envelope-Sender, X-Sender, Return-Path or a custom one like: X-hMailServer-Envelope-From before it gets transferred to SA to enable the SA SPF checking mescanism, as apparently SA isn't able to determine the MAIL FROM header on it's own resulting in the spf: cannot get Envelope-From, cannot use SPF debug message and thus, disable the SPF checks
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
RvdH
Senior user
Senior user
Posts: 608
Joined: 2008-06-27 14:42
Location: Netherlands

Re: SpamAssassin Custom Rules

Post by RvdH » 2015-07-19 18:59

LocalDelivery.cpp

Code: Select all

   bool 
   LocalDelivery::AddTraceHeaders_(std::shared_ptr<const Account> account, std::shared_ptr<Message> pMessage, const String &sOriginalAddress)
   {
      std::vector<std::pair<AnsiString, AnsiString> > fieldsToWrite;

      fieldsToWrite.push_back(std::make_pair("Return-Path", pMessage->GetFromAddress()));

      if (Configuration::Instance()->GetSMTPConfiguration()->GetAddDeliveredToHeader())
         fieldsToWrite.push_back(std::make_pair("Delivered-To", sOriginalAddress));

      const String fileName = PersistentMessage::GetFileName(account, pMessage);

      TraceHeaderWriter writer;
      return writer.Write(fileName, pMessage, fieldsToWrite);
   } 
So it seems Return-Path header is only added when delivered locally, and by the fact SorenR didn't have that Return-Path header I assume this is done right after the SA check completed

As far i can tell, this was handled differently before (hm 4.x), eg: viewtopic.php?p=16474#p16474
martin wrote:I looked at it a bit more. I've implemented it now so that hMailServer always adds a Return-path header to messages. But when it delivers a message to another server, it removes the Return-path header just before delivery.

I think this will work properly. SpamAssassin and other scripts will see the return-path headers. Messages downloaded using IMAP/POP3 will have a return-path header. When hMailServer delivers to an external server, it won't have a return-path header since the receiving server will add one..
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
SorenR
Senior user
Senior user
Posts: 2690
Joined: 2006-08-21 15:38
Location: Denmark

Re: SpamAssassin Custom Rules

Post by SorenR » 2015-07-20 00:02

RvdH wrote:So it seems Return-Path header is only added when delivered locally, and by the fact SorenR didn't have that Return-Path header I assume this is done right after the SA check completed
I believe this is right after the Account rules have been performed... It's sort of the very last thing done before releasing the email to the user.
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
RvdH
Senior user
Senior user
Posts: 608
Joined: 2008-06-27 14:42
Location: Netherlands

Re: SpamAssassin Custom Rules

Post by RvdH » 2015-07-20 08:52

SorenR wrote:
RvdH wrote:So it seems Return-Path header is only added when delivered locally, and by the fact SorenR didn't have that Return-Path header I assume this is done right after the SA check completed
I believe this is right after the Account rules have been performed... It's sort of the very last thing done before releasing the email to the user.
I just checked back on the 2 messages i got SPF_PASS hits when testing when i added the 'envelope_sender_header Return-Path' to local.cf... it seems these 2 messages already had 1 Return-Path header in them before it was passed to SA, after the final delivery they had 2 Return-Path headers

I now use 'envelope_sender_header From', i got quite some hits on SPF_PASS, SPF_NEUTRAL, SPF_SOFTFAIL etc... but i believe this isn't always the accurate value to use, it should use the value specified in Return-Path header, as this value can differ from the From header value
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
jimimaseye
Moderator
Moderator
Posts: 7624
Joined: 2011-09-08 17:48

Re: SpamAssassin Custom Rules

Post by jimimaseye » 2015-07-20 09:48

I must say I found it strange that when you added RETURN-PATH as the envelope header and got it to work because it didnt make sense given the findings about the placement of RETURN-PATH header. So no you have found that it already had one then that explains it.

As for using the FROM field, well I imagine this is hit and miss as a soltuion given that the FROM field can be spoofed to be anything or even blank.

Personally I think that HMS is wrong. If Spamassassin is to operate best on the full SMTP message received then it should be adding the final RETURN-PATH header (as it does) BEFORE it then passes it to Spamassassin. SA is to do its work based on everything about the message and is only adding its own headers (if configured to) and not doing anything to change the envelope or originating headers of the message. In other words, it should be the same as if HMS had accepted the message in its entirety and then the user passes it through to SA manually for checking - in this case the RETURN-PATH would already exist (quite right too). You should also add this is a bug for review to gituhub.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
RvdH
Senior user
Senior user
Posts: 608
Joined: 2008-06-27 14:42
Location: Netherlands

Re: SpamAssassin Custom Rules

Post by RvdH » 2015-07-20 10:03

Already done that, (#116)

Now I think of it, this also explains why my 'whitelist_from_spf' and 'whitelist_auth' rules failed to trigger most of the time, whitelist_auth also get triggered on DKIM and DOMAINKEYS checking
whitelist_auth user@example.com

Used to specify addresses which send mail that is often tagged (incorrectly) as spam. This is different from whitelist_from and whitelist_from_rcvd in that it first verifies that the message was sent by an authorized sender for the address, before whitelisting.

Authorization is performed using one of the installed sender-authorization schemes: SPF (using Mail::SpamAssassin::Plugin::SPF), or DKIM (using Mail::SpamAssassin::Plugin::DKIM). Note that those plugins must be active, and working, for this to operate.

Using whitelist_auth is roughly equivalent to specifying duplicate whitelist_from_spf, whitelist_from_dk, and whitelist_from_dkim lines for each of the addresses specified.

e.g.
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
mattg
Moderator
Moderator
Posts: 19257
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: SpamAssassin Custom Rules

Post by mattg » 2015-07-20 10:17

bypass on SPF pass is a bad idea....

In the hMailserver standard implementation of spf, any ~all (soft fail) will pass, as will any +all (refuse to pamper to spf).

Bill added in one of his alpha builds some ini switches that seem to work....

Code: Select all

SPFDefaultPolicy=v=spf1 MX A -all
; Set default SPF policy if sender does not have SPF
; Default is no default policy so don't use SPF

;SPFDefaultPolicyA=v=spf1 MX A ~all
; Set default SPF policy if sender does not have SPF but has A record
; Default is no default policy so don't use SPF

SPFPolicyOverride=v=spf1 -all
; Set SPF policy to override sender's policy such as block +all
; Default is do not override sender's policy
As you can see, I have uncommented some of the options.

Hard to test though without searching for spf records for hundreds of addresses, although my SPAM count seems low (currently 662 message, 1 virus, 0 SPAM) since last reboot 3 days ago, normally 1or2 SPAM per day on average, but normally more messages per day. (No SpamAssassin here)
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
RvdH
Senior user
Senior user
Posts: 608
Joined: 2008-06-27 14:42
Location: Netherlands

Re: SpamAssassin Custom Rules

Post by RvdH » 2015-07-20 10:19

mattg wrote:bypass on SPF pass is a bad idea....

In the hMailserver standard implementation of spf, any ~all (soft fail) will pass, as will any +all (refuse to pamper to spf).

Bill added in one of his alpha builds some ini switches that seem to work....

Code: Select all

SPFDefaultPolicy=v=spf1 MX A -all
; Set default SPF policy if sender does not have SPF
; Default is no default policy so don't use SPF

;SPFDefaultPolicyA=v=spf1 MX A ~all
; Set default SPF policy if sender does not have SPF but has A record
; Default is no default policy so don't use SPF

SPFPolicyOverride=v=spf1 -all
; Set SPF policy to override sender's policy such as block +all
; Default is do not override sender's policy
As you can see, I have uncommented some of the options.

Hard to test though without searching for spf records for hundreds of addresses, although my SPAM count seems low (currently 662 message, 1 virus, 0 SPAM) since last reboot 3 days ago, normally 1 SPAM per day on average, but normally closer to 600 messages per day too
of course, i only bypass on spf for messages that are often faultly marked as SPAM
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
jimimaseye
Moderator
Moderator
Posts: 7624
Joined: 2011-09-08 17:48

Re: SpamAssassin Custom Rules

Post by jimimaseye » 2015-07-20 10:27

RvdH wrote:Already done that, (#116)
Ah yes, I see now.

However, for clarity I think the clear problem is the lack of adding the RETURN-PATH to the message after full receipt of the SMTP delivery and BEFORE passing to SA. The github report talks about specifying 'pseudo-headers' to local.cf but that still doesnt fix the problem as the available choice of headers that the user can select from are not the best (for reasons discussed above) and RETURN-PATH is the most logical and suitable one. HMS needs to complete the message with this header for passing to SA (then no user-config to local.cf would be necessary). After all, EVERY message must have a RETURN-PATH, always.

(Note, be aware that this post was done more for the benefit of Martin/readers following on from the github report)
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
RvdH
Senior user
Senior user
Posts: 608
Joined: 2008-06-27 14:42
Location: Netherlands

Re: SpamAssassin Custom Rules

Post by RvdH » 2015-07-20 10:31

No, i think you read it wrong, SA uses a internal EnvelopeFrom pseudo-header, it can be passed in different ways, eg:

using the common names: (SA automatically checks these)
X-Envelope-From, Envelope-Sender, X-Sender or Return-Path

using a custom name: (you must provide the appropriate header to SA)
X-hMailServer-Envelope-From


Although, i agree with you that adding the Return-Path header before sending it to SA, seems like the preferred way to fix this, as a message always MUST have a Return-Path header before "final delivery" according to RFC2821, But there is a glitch in your assumption of the Return-Path usage, the magic word in that sentence is "final delivery", eg:
RFC2821:
When the delivery SMTP server makes the "final delivery" of a
message, it inserts a return-path line at the beginning of the mail
data. This use of return-path is required; mail systems MUST support
it. The return-path line preserves the information in the <reverse-
path> from the MAIL command. Here, final delivery means the message
has left the SMTP environment. Normally, this would mean it had been
delivered to the destination user or an associated mail drop, but in
some cases it may be further processed and transmitted by another
mail system.
So basically, the way hMailServer handles this is the the appropriate way...it adds that header just before "final delivery" when performing a Local Delivery
That said, maybe it is better to use X-Envelope-From, Envelope-Sender, X-Sender or a custom one like X-hMailServer-Envelope-From
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
jimimaseye
Moderator
Moderator
Posts: 7624
Joined: 2011-09-08 17:48

Re: SpamAssassin Custom Rules

Post by jimimaseye » 2015-07-20 12:06

I havent misunderstood. The final delivery will be the conclusion of the sending SMTP data from the sending smtp server, not when HMS delivers it to an account. As all of the information has been received from the sending server by the time that HMS then sends it to SA it can add the return-path as there will be no other modifications to the SMTP envelope or data from that stage (specifically the "MAIL FROM" which is what 'return-path' is populated from). It will either then be accepted as it is or rejected (due to spam checks). After all, if it fails an internal HMS spam check (assuming you dont use spamassassin) that return-path will still be there.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
RvdH
Senior user
Senior user
Posts: 608
Joined: 2008-06-27 14:42
Location: Netherlands

Re: SpamAssassin Custom Rules

Post by RvdH » 2015-07-20 12:26

But if HM delivers a message to another server, it should remove the Return-path header if i correctly understand what martin wrote on 4.x release posted earlier

In this case this mean that HM first need to add it and later remove it, isn't it easier to use one of the alternative headers then?
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
jimimaseye
Moderator
Moderator
Posts: 7624
Joined: 2011-09-08 17:48

Re: SpamAssassin Custom Rules

Post by jimimaseye » 2015-07-20 13:52

Yes that is correct. That is certainly how it (supposed to) work/reads (from Martins quote).

a) SENDING Server passes WITHOUT return-path to a RECEIVING hms server
b) HMS received from sending server and ADDS return-path.
c) HMS passes to SA (or whatever other spam checks it wants to do)
d) HMS (is either then to deliver locally or ) passes out to another SMTP server
d1-- locally = all is in place (including return-path))
d2-- to SMTP server= return-path is removed (as quoted by martin)
a) RECEIVING smtp server receives email (without the return-path)
b) Receiving SMTP server adds its own RETURN-PATH header
etc

If things were working like this with the return-path (ie the RETURN-PATH in place prior to calling to SA), as Martins suggestion, then SA wouldnt need any extra config as by default it would see the RETURN-PATH and its SPF checks would work... just as they already do with emails received by External Download.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
RvdH
Senior user
Senior user
Posts: 608
Joined: 2008-06-27 14:42
Location: Netherlands

Re: SpamAssassin Custom Rules

Post by RvdH » 2015-07-21 10:28

Makes me wonder why martin changed that Return-Path behavior between 4.x and the 5.x version of HMS, as it reads from his quote he was aware of the Return-Path requirement of SA back in the days...
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
jimimaseye
Moderator
Moderator
Posts: 7624
Joined: 2011-09-08 17:48

Re: SpamAssassin Custom Rules

Post by jimimaseye » 2015-07-21 11:37

Yes I agree. From what he has written, it seems he INTENDED it to be as I explained above (which IMO is the correct thing to do). But he has mistimed it: that is to say he is adding the RETURN-PATH at the end ready for going to SA (as we want it to be):
Martin wrote: But when it delivers a message to another server, it removes the Return-path header just before delivery.

I think this will work properly. SpamAssassin and other scripts will see the return-path headers
...but the problem is that this isnt what is happening because Spamassassin checks are being done BEFORE he is adding the header and not as he (has written it to be) intended.

(But yes, like you I do wonder why he chose to change to this (intended) behaviour from 4 to 5 - maybe to make the above work. He just needs to do it :-) )

EDIT: Actually it gives more light when reading that thread your quoted martin from.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
RvdH
Senior user
Senior user
Posts: 608
Joined: 2008-06-27 14:42
Location: Netherlands

Re: SpamAssassin Custom Rules

Post by RvdH » 2015-07-22 15:58

SorenR wrote:
jimimaseye wrote:
SorenR wrote:Hmm... I did some tests yesterday and also intercepted the returnfile from SpamAssassin...

NO "Return-Path: sender@domain.com" to be found anywhere in the email... Weird as it IS there when email is delivered into INBOX... :?
Thats an interesting find. And actually goes some to explain my test results: even WITH the 'envelope_sender_header Return-Path' parameter added to local.cf (as OP said he did and got a result), it still didnt make a difference for me (and yet the 'return-path' header is definitely in evidence in the message when you view it). If the header is available to SA (as you say) then it isnt going to use it.

So, with or without "envelope_sender_header Return-Path" I never got the SPF_PASS check direct SMTP deliveries and yet on an External Download it is always there (because I assume SA is able to read the 'return path' header as it will already have been completed and filled out).
Presumable...

I tried to add it to the headers under the OnSMTPData trigger with oMessage.Headervalue("Return-Path") = oMessage.FromAddress and oMessage.Save... Made a complete FU on all the headers :oops:
I tried it like this (see below), but this doesn't seems to fix anything...i guess it expects the Return-Path header as topmost header

Code: Select all

Sub OnAcceptMessage(oClient, oMessage)
If ((oClient.Port = 25) Or (oClient.Port = 587) Or (oClient.Port = 465)) Then     
	dim FromAddress
	On Error Resume Next
	If oMessage.FromAddress <> "" then
		FromAddress = Split(oMessage.FromAddress,"@")(0) + "@" + Split(oMessage.FromAddress,"@")(1) 
	end if
	if InStr(FromAddress,"@")=0 then
		If oMessage.From <> "" then
			FromAddress = Split(oMessage.From,"@")(0) + "@" + Split(oMessage.From,"@")(1) 
		end if
	end if
	If FromAddress <> "" And InStr(FromAddress,"@")<>0 then
		EventLog.Write("Return-Path=" & FromAddress)
		oMessage.Headervalue("Return-Path") = FromAddress
		oMessage.Headervalue("X-Envelope-From") = FromAddress
		oMessage.Save
	end if
	On Error Resume Next
end if
End Sub
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
SorenR
Senior user
Senior user
Posts: 2690
Joined: 2006-08-21 15:38
Location: Denmark

Re: SpamAssassin Custom Rules

Post by SorenR » 2015-07-22 16:33

email is sent to SA before OnAcceptMessage... :wink:
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
RvdH
Senior user
Senior user
Posts: 608
Joined: 2008-06-27 14:42
Location: Netherlands

Re: SpamAssassin Custom Rules

Post by RvdH » 2015-07-22 16:37

Oops :oops:

But you were right, when done in OnSMTPData the message get FU badly!
Last edited by RvdH on 2015-07-22 17:03, edited 2 times in total.
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
jimimaseye
Moderator
Moderator
Posts: 7624
Joined: 2011-09-08 17:48

Re: SpamAssassin Custom Rules

Post by jimimaseye » 2015-07-22 16:51

SorenR wrote:email is sent to SA before OnAcceptMessage... :wink:
Exactly. That was the point of my explanation above. He isnt adding RETURN-PATH early enough (and does the SA checks as part of the SMTP envelope data transmission instead of at the end of it) which is contrary to what was quoted by Martin in that thread. I really think this is a flaw and Martin should be aware of it (and maybe comment why he changed and thought it was working on that thread).
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
RvdH
Senior user
Senior user
Posts: 608
Joined: 2008-06-27 14:42
Location: Netherlands

Re: SpamAssassin Custom Rules

Post by RvdH » 2015-07-29 13:33

Martin?
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
jimimaseye
Moderator
Moderator
Posts: 7624
Joined: 2011-09-08 17:48

Re: SpamAssassin Custom Rules

Post by jimimaseye » 2016-01-04 22:43

RvdH wrote:

Code: Select all

X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) * on Server * at Sun, 19
 Jul 2015 08:13:19 +0200
X-Spam-Status: Yes, score=5.4, hits=5.4, required=4.0, autolearn=no autolearn_force=no,
 shortcircuit=no
X-Spam-Level: *****
X-Spam-Report: *  0.7 SPF_SOFTFAIL SPF: sender does not match SPF record (softfail) *  1.5
 SUBJ_ALL_CAPS Subject is all capitals *  0.2 JAM_LONG_LINK BODY: Very long
 link in mail, possibly filled up with *      random words by bulk mailer *
 0.4 HTML_IMAGE_RATIO_02 BODY: HTML has a low ratio of text to image area *
  0.0 HTML_MESSAGE BODY: HTML included in message *  1.0 HTML_FONT_FACE_BAD
 BODY: HTML font face is not a word *  0.8 BAYES_50 BODY: Bayes spam
 probability is 40 to 60% *      [score: 0.4965] *  0.8 MPART_ALT_DIFF
 BODY: HTML and text parts are different
...
X-hMailServer-Spam: YES
X-hMailServer-Reason-1: Tagged as Spam by SpamAssassin - (Score: 5)
X-hMailServer-Reason-Score: 5
As you can see the X-Spam-Report header markup is gone, looks like line breaks are missing or something

Without the X-hMailServer headers, it looks ok, eg:

Code: Select all

X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) * on
	Server * at Sun, 19 Jul 2015 08:13:19 +0200
X-Spam-Status: No, score=-7.1, hits=-7.1, required=4.0, autolearn=no
	autolearn_force=no, shortcircuit=no
X-Spam-Report: 
	* -6.0 USER_IN_WHITELIST_TO User is listed in 'whitelist_to'
	* -0.0 SPF_PASS SPF: sender matches SPF record
	*  0.8 PDF_ATTACHED BODY: email contains a PDF file attachment
	* -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%
	*      [score: 0.0000]
Do you use hMailServer or SpamAssassin to alter the message subject? (eg: prefix it with [SPAM])

I notice a difference with my result, eg:
Subject: [SPAM] [5.1] P11Ds for the tax year 2014/15
X-Spam-Prev-Subject: P11Ds for the tax year 2014/15
https://github.com/hmailserver/hmailserver/issues/138
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
jimimaseye
Moderator
Moderator
Posts: 7624
Joined: 2011-09-08 17:48

Re: SpamAssassin Custom Rules

Post by jimimaseye » 2016-02-26 23:38

Sorry to come back to this.... (I am doing so because today for a week or so I am turning on Direct deliveries in instead of using External Download)....

Although we acknowledge that the SPF test is failing due to the lack of RETURN-PATH header at time of being passed to SA, and that adding "envelope_sender_header From " in to local.cf gives it something to check and therefore report on (albeit not too guaranteed relialke due to blank or spoofing opportunities), but really.... is it worth it?! Ive just noticed that despite it all,the SPF_PASS check that spamassassin does still only scores it as ZERO for a positive pass.

Code: Select all

	* -0.0 SPF_PASS SPF: sender matches SPF record
Its true, that a fail might give a positive score (increasing the scoring towards a 'spam' conclusion), and given that the from address frankly could be anything and is possible to result in a positive score under these circumstances, it simply cant be trusted to bring any genuine reliable value to the equation.

I think its best to be left out (in absence of the RETURN-PATH field)
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
mattg
Moderator
Moderator
Posts: 19257
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: SpamAssassin Custom Rules

Post by mattg » 2016-02-27 00:49

You can change the score if you want.

Also be careful that SpamAssassin doesn't pass SPF where the SPF record ends in a '+all'

+all means that ANY IP address meets SPF.
and '~all' is a softfail = means OK we have noted that it is different, but that's OK

In my view unless the SPF ends in a '-all' the SPF check should always fail
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
SorenR
Senior user
Senior user
Posts: 2690
Joined: 2006-08-21 15:38
Location: Denmark

Re: SpamAssassin Custom Rules

Post by SorenR » 2016-02-27 01:17

Currently trying out this...

spf-test.cf

Code: Select all

# SPF THAT DOESN'T REALLY CARE IF EMAIL IS A FORGERY
ifplugin Mail::SpamAssassin::Plugin::AskDNS
  askdns   JMQ_SPF_NEUTRAL_ALL _SENDERDOMAIN_ TXT /^v=spf1 .+\?all$/
  describe JMQ_SPF_NEUTRAL_ALL SPF set to ?all!
  score    JMQ_SPF_NEUTRAL_ALL 0.5
endif
http://cpansearch.perl.org/src/KMCGRAIL ... /AskDNS.pm

I have dropped both in c:\SpamAssassin\etc\spamassassin
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
jimimaseye
Moderator
Moderator
Posts: 7624
Joined: 2011-09-08 17:48

Re: SpamAssassin Custom Rules

Post by jimimaseye » 2016-02-27 01:46

SorenR wrote:Currently trying out this...

spf-test.cf

Code: Select all

# SPF THAT DOESN'T REALLY CARE IF EMAIL IS A FORGERY
ifplugin Mail::SpamAssassin::Plugin::AskDNS
  askdns   JMQ_SPF_NEUTRAL_ALL _SENDERDOMAIN_ TXT /^v=spf1 .+\?all$/
  describe JMQ_SPF_NEUTRAL_ALL SPF set to ?all!
  score    JMQ_SPF_NEUTRAL_ALL 0.5
endif
http://cpansearch.perl.org/src/KMCGRAIL ... /AskDNS.pm

I have dropped both in c:\SpamAssassin\etc\spamassassin
Yeah but does it still depend on the RETURN-PATH being sent to SA for use in the SPF check. Without it you still need to state a "envelope_sender_header from" setting in local.cf. The point is that the weakness is that the FROM address is being tested (not that SPF checking is doing its job robustly enough).
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
jimimaseye
Moderator
Moderator
Posts: 7624
Joined: 2011-09-08 17:48

Re: SpamAssassin Custom Rules

Post by jimimaseye » 2016-02-27 12:32

mattg wrote:You can change the score if you want.

Also be careful that SpamAssassin doesn't pass SPF where the SPF record ends in a '+all'

+all means that ANY IP address meets SPF.
and '~all' is a softfail = means OK we have noted that it is different, but that's OK

In my view unless the SPF ends in a '-all' the SPF check should always fail
I think spamassassin already acknowledge that checking SPF records is a pointless exercise (due to the inadequacies of using SPF for spam testing) and make remarks as such (as well as effectively scoring zero for all tests).

from 50_scores.cf:

Code: Select all

# SPF
# Note that the benefit for a valid SPF record is deliberately minimal; it's
# likely that more spammers would quickly move to setting valid SPF records
# otherwise.  The penalties for an *incorrect* record, however, are large.  ;)
ifplugin Mail::SpamAssassin::Plugin::SPF
score SPF_NONE 0
score SPF_HELO_NONE 0
score SPF_PASS -0.001
score SPF_HELO_PASS -0.001
# <gen:mutable>
score SPF_FAIL 0 0.919 0 0.001 # n=0 n=2
score SPF_HELO_FAIL 0 0.001 0 0.001 # n=0 n=2
score SPF_HELO_NEUTRAL 0 0.001 0 0.112 # n=0 n=2
score SPF_HELO_SOFTFAIL 0 0.896 0 0.732 # n=0 n=2
score SPF_NEUTRAL 0 0.652 0 0.779 # n=0 n=2
score SPF_SOFTFAIL 0 0.972 0 0.665 # n=0 n=2
OF course, as you say, if the user puts more value and reliance on the tests then they are free to adjust the scores to suit themselves.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
mattg
Moderator
Moderator
Posts: 19257
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: SpamAssassin Custom Rules

Post by mattg » 2016-02-27 16:48

But none of that really looks at the issue of SPF ending in a +all

http://wiki.apache.org/spamassassin/Rules/SPF_NEUTRAL
A "Neutral" result MUST be treated exactly like the "None" result; the distinction exists only for informational purposes.
I disagree with that statement, and there is no SPF_None option
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
SorenR
Senior user
Senior user
Posts: 2690
Joined: 2006-08-21 15:38
Location: Denmark

Re: SpamAssassin Custom Rules

Post by SorenR » 2016-02-27 17:27

jimimaseye wrote:
SorenR wrote:Currently trying out this...

spf-test.cf

Code: Select all

# SPF THAT DOESN'T REALLY CARE IF EMAIL IS A FORGERY
ifplugin Mail::SpamAssassin::Plugin::AskDNS
  askdns   JMQ_SPF_NEUTRAL_ALL _SENDERDOMAIN_ TXT /^v=spf1 .+\?all$/
  describe JMQ_SPF_NEUTRAL_ALL SPF set to ?all!
  score    JMQ_SPF_NEUTRAL_ALL 0.5
endif
http://cpansearch.perl.org/src/KMCGRAIL ... /AskDNS.pm

I have dropped both in c:\SpamAssassin\etc\spamassassin
Yeah but does it still depend on the RETURN-PATH being sent to SA for use in the SPF check. Without it you still need to state a "envelope_sender_header from" setting in local.cf. The point is that the weakness is that the FROM address is being tested (not that SPF checking is doing its job robustly enough).
Nope... It only depend on the plugin AskDNS... It's checking the SPF TXT record in DNS ;-)

If it works at all... (?) I'll let it run for a few days and see if I get something.

The rule originate from the KAM.cf ruleset.
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
SorenR
Senior user
Senior user
Posts: 2690
Joined: 2006-08-21 15:38
Location: Denmark

Re: SpamAssassin Custom Rules

Post by SorenR » 2016-02-27 18:00

SorenR wrote:Nope... It only depend on the plugin AskDNS... It's checking the SPF TXT record in DNS ;-)

If it works at all... (?) I'll let it run for a few days and see if I get something.

The rule originate from the KAM.cf ruleset.
I hate quoting myself but I could not edit above...

New ruleset I'm testing...

Code: Select all

ifplugin Mail::SpamAssassin::Plugin::AskDNS

  askdns   SR_SPF_PASS_ALL _SENDERDOMAIN_ TXT /^v=spf1 .+\+all$/
  describe SR_SPF_PASS_ALL SPF set to +all!
  score    SR_SPF_PASS_ALL 0.0

  askdns   SR_SPF_FAIL_ALL _SENDERDOMAIN_ TXT /^v=spf1 .+\-all$/
  describe SR_SPF_FAIL_ALL SPF set to -all!
  score    SR_SPF_FAIL_ALL 0.0

  askdns   SR_SPF_SOFTFAIL_ALL _SENDERDOMAIN_ TXT /^v=spf1 .+\~all$/
  describe SR_SPF_SOFTFAIL_ALL SPF set to ~all!
  score    SR_SPF_SOFTFAIL_ALL 0.0

  askdns   SR_SPF_NEUTRAL_ALL _SENDERDOMAIN_ TXT /^v=spf1 .+\?all$/
  describe SR_SPF_NEUTRAL_ALL SPF set to ?all!
  score    SR_SPF_NEUTRAL_ALL 0.0

endif
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
SorenR
Senior user
Senior user
Posts: 2690
Joined: 2006-08-21 15:38
Location: Denmark

Re: SpamAssassin Custom Rules

Post by SorenR » 2016-03-02 01:20

[quote="SorenR"]New ruleset I'm testing...

Code: Select all

ifplugin Mail::SpamAssassin::Plugin::AskDNS

  askdns   SR_SPF_PASS_ALL _SENDERDOMAIN_ TXT /^v=spf1 .+\+all$/
  describe SR_SPF_PASS_ALL SPF set to +all!
  score    SR_SPF_PASS_ALL 0.0

  askdns   SR_SPF_FAIL_ALL _SENDERDOMAIN_ TXT /^v=spf1 .+\-all$/
  describe SR_SPF_FAIL_ALL SPF set to -all!
  score    SR_SPF_FAIL_ALL 0.0

  askdns   SR_SPF_SOFTFAIL_ALL _SENDERDOMAIN_ TXT /^v=spf1 .+\~all$/
  describe SR_SPF_SOFTFAIL_ALL SPF set to ~all!
  score    SR_SPF_SOFTFAIL_ALL 0.0

  askdns   SR_SPF_NEUTRAL_ALL _SENDERDOMAIN_ TXT /^v=spf1 .+\?all$/
  describe SR_SPF_NEUTRAL_ALL SPF set to ?all!
  score    SR_SPF_NEUTRAL_ALL 0.0

endif
Bummer, crap, bloody hell, )(///&%#¤¤#%#%"¤!"

_SENDERDOMAIN_ is a new TAG template from version 3.4.1.
Jam-software only support 3.4.0 and for some reason I cannot compile SpamAssassin 3.4.1 according to this... https://wiki.apache.org/spamassassin/In ... gOnWindows :evil:
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
mattg
Moderator
Moderator
Posts: 19257
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: SpamAssassin Custom Rules

Post by mattg » 2016-03-02 01:57

I've tried this, but I'm not sure if it is working or not.
I find SpamAssassin really hard to test

This has been added to my local.cf


score SPF_SOFTFAIL 1
score SPF_NEUTRAL 3
score SPF_PASS -1
score SPF_FAIL 6
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
SorenR
Senior user
Senior user
Posts: 2690
Joined: 2006-08-21 15:38
Location: Denmark

Re: SpamAssassin Custom Rules

Post by SorenR » 2016-03-03 20:15

mattg wrote:I've tried this, but I'm not sure if it is working or not.
I find SpamAssassin really hard to test

This has been added to my local.cf


score SPF_SOFTFAIL 1
score SPF_NEUTRAL 3
score SPF_PASS -1
score SPF_FAIL 6
I'm messing with some logging and came across this...

Code: Select all

'   Dim WshShell, oExec
'   Set WshShell = WScript.CreateObject("WScript.Shell")
'
'   Set oExec = WshShell.Exec('nslookup -type=TXT google.com | find "v="')
'   -- or --
'   Set oExec = WshShell.Exec('dig google.com TXT | find "v="')
'
'   x = oExec.StdOut.ReadLine
'   Wscript.Echo x
Perhaps it can be usefull at some point... :?:
Above are notes, have not tried to run it yet..
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
RvdH
Senior user
Senior user
Posts: 608
Joined: 2008-06-27 14:42
Location: Netherlands

Re: SpamAssassin Custom Rules

Post by RvdH » 2016-03-03 21:49

Version 3.4.1.36: (03 March 2016)
•Apache SpamAssassin version 3.4.1 has been integrated.
•ActivePerl 5.8 has been replaced by Strawberry Perl 5.22.
•Native 64 bit support has been integrated.
•Support for Windows XP and Server 2003 has been dropped.
•A subfolder runtime, containing the Perl scripts sa-learn, sa-update, spamassassin, spamd, and their dependencies has been added.
•Known issue: The installation path and arguments at the command prompt must only contain ASCII characters.
•Known issue: Due to a memory leak in the SpamAssassin Daemon (SpamD), a periodical reboot of the process is required. For professional use we recommend our service-based solution "SpamAssassin in a Box".

https://www.jam-software.com/spamassassin/changes.shtml
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
SorenR
Senior user
Senior user
Posts: 2690
Joined: 2006-08-21 15:38
Location: Denmark

Re: SpamAssassin Custom Rules

Post by SorenR » 2016-03-03 21:54

RvdH wrote:Version 3.4.1.36: (03 March 2016)
•Apache SpamAssassin version 3.4.1 has been integrated.
•ActivePerl 5.8 has been replaced by Strawberry Perl 5.22.
•Native 64 bit support has been integrated.
•Support for Windows XP and Server 2003 has been dropped.
•A subfolder runtime, containing the Perl scripts sa-learn, sa-update, spamassassin, spamd, and their dependencies has been added.
•Known issue: The installation path and arguments at the command prompt must only contain ASCII characters.
•Known issue: Due to a memory leak in the SpamAssassin Daemon (SpamD), a periodical reboot of the process is required. For professional use we recommend our service-based solution "SpamAssassin in a Box".

https://www.jam-software.com/spamassassin/changes.shtml
Wohoo - That was about bloody time :mrgreen:

"Support for Windows XP and Server 2003 has been dropped." Bummer :evil:
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
jimimaseye
Moderator
Moderator
Posts: 7624
Joined: 2011-09-08 17:48

Re: SpamAssassin Custom Rules

Post by jimimaseye » 2016-03-03 22:22

"Support for Windows XP and Server 2003 has been dropped." Bummer
Doesnt mean it wont work though, it just means they wont make an effort to ensure it does. I guess you just need to try it and see.

Dont like this, though:
•A subfolder runtime, containing the Perl scripts sa-learn, sa-update, spamassassin, spamd, and their dependencies has been added.
Means being careful when upgrading. Not sure a 0.0.1 version increase is enough for me to worry about upgrading to considering the effort required (if it aint broke etc etc). Mind you, that's what I think today. Tomorrow might be different.)
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
jimimaseye
Moderator
Moderator
Posts: 7624
Joined: 2011-09-08 17:48

Re: SpamAssassin Custom Rules

Post by jimimaseye » 2016-03-03 23:04

From http://spamassassin.apache.org/news.html
News and Announcements Atom feed

2015-04-30: SpamAssassin 3.4.1 has been released! Highlights include:
  • * improved automation to help combat spammers that are abusing new top level domains;
    * tweaks to the SPF support to block more spoofed emails;
    * increased character set normalization to make rules easier to develop and stop spammers from using alternate character sets to bypass tests;
    * continued refinement to the native IPv6 support; and
    * improved Bayesian classification with better debugging and attachment hashing.
Only "tweaks" to SPF support (at least it will give Soren the chance to try that rule out above).
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
SorenR
Senior user
Senior user
Posts: 2690
Joined: 2006-08-21 15:38
Location: Denmark

Re: SpamAssassin Custom Rules

Post by SorenR » 2016-03-03 23:05

jimimaseye wrote:
"Support for Windows XP and Server 2003 has been dropped." Bummer
Doesnt mean it wont work though, it just means they wont make an effort to ensure it does. I guess you just need to try it and see.
Function "strcpy_s" appeared first in VC8 and when I start spamd it says "The procedure entry point strcpy_s could not be located in the dynamic link library msvcrt.dll"
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

Post Reply