SpamAssassin Questions

Use this forum for discussions about SpamAssassin and anti-spam in general.
thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

SpamAssassin Questions

Post by thomas10 » 2018-02-02 10:30

Hi All,

Recently, the mail server here was attacked by Chinese spam mail with domain name @qq.com, @qq.ocm, @123.com, @163.com, etc. I can't straight set rules to delete emails from those domains as some emails are genuine.

This mail server doesn't use SpamAssassin, so I plan to move current Hmail to other PC and set SA there. I am using latest Jam Software SpamAssassin Windows.
I referred to below link on the instruction to setup SA:
viewtopic.php?p=164408
viewtopic.php?f=21&t=28133

Below are my questions:
1. I have manually separate the Spam and Ham and train bayes database via trainbayes.bat file. (Few thousands of the Chinese spam mail and 3 are ham.) Will it be able to mark the chinese spam mail score and delete them accordingly?

2. For global rules setting:
a) I want to forward emails contains [Spam] in subject to another email so I can check the mail whether spam or ham. Any idea?

3. Do you have any customs .cf file that configured to protect against Chinese spam mail?

4. In local.cf, I set use_bayes 1, bayes_auto_learn 0, report_safe 0, required_score 5.0, rewrite_header Subject [_HITS_].
a) May I know what's the use of add_header all Report _REPORT_ *?
b) Is it ok for me to set the bayes_auto_learn as 0 because I am scared on false positive email?
c) If found false positive, what should I do? Should I get the eml file to let SA learn or just set whitelist_from in local.cf?

5. Can I have multiple .cf files for SA Windows version? Because I was thinking to set whitelist/ blacklist rule on separate cf file.

6. How to make spamd log rotate daily? I read some users make it along when performing sa-update, does it work? (viewtopic.php?f=21&t=28133#p183044)

7. I read some of the thread mentioning KAM.cf updates which is still good, how can I set to download for sa-update?

8. For Hmail setting, I plan to follow Mr jimimaseye way by setting Use SA score: false-
5, Hmail spam mark threshold-5 but delete threshold- 20. Under SpamAssassin tab, what should I put for "Host Name"? localhost or 127.0.0.1? I set SA on the same PC that installs Hmail.

9. I have around 300 users and the email flow is quite fast. If SA is being set to use, normally how long will SA take to process an email?

I am so sorry for such a long list, but really appreciated if can settle this one. Cheers guys and sorry for trouble.

Regards,
Thomas

User avatar
jimimaseye
Moderator
Moderator
Posts: 8172
Joined: 2011-09-08 17:48

Re: SpamAssassin Questions

Post by jimimaseye » 2018-02-02 14:56

thomas10 wrote: Below are my questions:
1. I have manually separate the Spam and Ham and train bayes database via trainbayes.bat file. (Few thousands of the Chinese spam mail and 3 are ham.) Will it be able to mark the chinese spam mail score and delete them accordingly?
Spamassassin has a rule that scores BAYES based 'probablity of spam' accordingly. Of course it also also looks at a lot of other things too and the final score will be a combination of it.
thomas10 wrote: 2. For global rules setting:
a) I want to forward emails contains [Spam] in subject to another email so I can check the mail whether spam or ham. Any idea?
From viewtopic.php?f=21&t=28133 as an example
2, Set a GLOBAL RULE:

Name: "ExternalScore7"
if:
X-Spam-Level (custom header) ...contains..... ******* <<-- 7 asterix
action:
Move to IMAP Folder...... Trash (or whatever your normal trash folder is on your accounts (you may choose a dedicated 'Spam' folder instead). Alternatively you can simply 'DELETE EMAIL' but that never gives you chance to review.)
You can change the action to FORWARD EMAIL if you wish.
a) May I know what's the use of add_header all Report _REPORT_ *?
Adds the Spamassassin headers (scoring results) to all emails (if report_safe is 0) https://spamassassin.apache.org/full/3. ... _Conf.html
b) Is it ok for me to set the bayes_auto_learn as 0 because I am scared on false positive email?
Youre free to do what you wish. Bayes is good though. with careful scoriung and rules you can manage FP's and at least mark FP's as 'potential' spam (requiring review) rather than out-right deletion as 'definite' spam.
c) If found false positive, what should I do? Should I get the eml file to let SA learn or just set whitelist_from in local.cf?
Retrain your bayes. viewtopic.php?f=20&t=26866
5. Can I have multiple .cf files for SA Windows version? Because I was thinking to set whitelist/ blacklist rule on separate cf file.
You can have as many as you want just like in the linux environment. (Windows version is designed to be the same as the linux version)
6. How to make spamd log rotate daily? I read some users make it along when performing sa-update, does it work? (viewtopic.php?f=21&t=28133#p183044)
No answer required.
Hmail spam mark threshold-5 but delete threshold- 20. Under SpamAssassin tab, what should I put for "Host Name"? localhost or 127.0.0.1? I set SA on the same PC that installs Hm
If spamassassin is running on the SAME MACHINE as you Hmailserver then yes you can put localhost. Otherwise state the IP address of the machine you have spamassassin running on.
9. I have around 300 users and the email flow is quite fast. If SA is being set to use, normally how long will SA take to process an email?
Depends on yor setup ut on average each email could take between 0.5 and 1 second. ut no guarantees as it depends on your processor, DNS config and the internet!
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-05 08:46

Thank you so much jimimaseye
thomas10 wrote: 2. For global rules setting:
a) I want to forward emails contains [Spam] in subject to another email so I can check the mail whether spam or ham. Any idea?
From viewtopic.php?f=21&t=28133 as an example
2, Set a GLOBAL RULE:

Name: "ExternalScore7"
if:
X-Spam-Level (custom header) ...contains..... ******* <<-- 7 asterix
action:
Move to IMAP Folder...... Trash (or whatever your normal trash folder is on your accounts (you may choose a dedicated 'Spam' folder instead). Alternatively you can simply 'DELETE EMAIL' but that never gives you chance to review.)
You can change the action to FORWARD EMAIL if you wish.
My users are using pop3 instead of IMAP, so move to IMAP folder should not work. Since this is the first time using SA, I think to forward any emails with subject [SPAM] to spamcatch account in case there is FP.
b) Is it ok for me to set the bayes_auto_learn as 0 because I am scared on false positive email?
Youre free to do what you wish. Bayes is good though. with careful scoriung and rules you can manage FP's and at least mark FP's as 'potential' spam (requiring review) rather than out-right deletion as 'definite' spam.
Ok, guess I will try to use with bayes_auto_learn 1 to see how's it going.
c) If found false positive, what should I do? Should I get the eml file to let SA learn or just set whitelist_from in local.cf?
Retrain your bayes. viewtopic.php?f=20&t=26866
Talk about retrain, does it mean I need to remove the bayes database and retrain again? or just get the eml file to train it as Ham?

For Hmail antispam setting, I am going for jimimaseye way:
General
Spam Mark-5 ( all selected)
Spam Delete threshold- 20

Spam Test:
SPF:True- 3
Check HELO host: True- 2
Check MX records: True- 2
Verify DKIM: False- 5

Spamassassin:
Use SA: True
Hostname: localhost
Port: 783
Use SA score: False- 5

But how about DNSBL and SURBL entries? My current Hmail server DNSBL and SURBL entries are as below. Not sure whether this is enough?
DNSBL ENTRIES:
zen.spamhaus.org- Score: 3 Result: 127.0.0.2-8|127.0.0.10-11
bl.spamcop.net- Score: 3 Result: 127.0.0.2

SURBL ENTRIES:
multi.surbl.org- Score: 3


Please have a look on my local.cf to see where did I miss out. Thanks so much again.

Code: Select all

# This is the right place to customize your installation of SpamAssassin.
#
# See 'perldoc Mail::SpamAssassin::Conf' for details of what can be
# tweaked.
#
# Only a small subset of options are listed below
#
###########################################################################

#   Add *****SPAM***** to the Subject header of spam e-mails
#
rewrite_header Subject [_HITS_]


#   Save spam messages as a message/rfc822 MIME attachment instead of
#   modifying the original message (0: off, 2: use text/plain instead)
#
report_safe 0
add_header all Report_REPORT_*

#   Set which networks or hosts are considered 'trusted' by your mail
#   server (i.e. not spammers)
#
# trusted_networks 212.17.35.


#   Set file-locking method (flock is not safe over NFS, but is faster)
#
# lock_method flock


#   Set the threshold at which a message is considered spam (default: 5.0)
#
required_score 5.0


#   Use Bayesian classifier (default: 1)
#
use_bayes 1


#   Bayesian classifier auto-learning (default: 1)
#
bayes_auto_learn 1


#   Set headers which may provide inappropriate cues to the Bayesian
#   classifier
#
bayes_ignore_header X-Bogosity
bayes_ignore_header X-Spam-Flag
bayes_ignore_header X-Spam-Status
bayes_ignore_header X-Complaints-To
bayes_ignore_header X-Abuse
bayes_ignore_header X-Report-Abuse
bayes_ignore_header X-AntiAbuse


#   Whether to decode non- UTF-8 and non-ASCII textual parts and recode
#   them to UTF-8 before the text is given over to rules processing.
#
# normalize_charset 1

#   Some shortcircuiting, if the plugin is enabled
# 
ifplugin Mail::SpamAssassin::Plugin::Shortcircuit
#
#   default: strongly-whitelisted mails are *really* whitelisted now, if the
#   shortcircuiting plugin is active, causing early exit to save CPU load.
#   Uncomment to turn this on
#
# shortcircuit USER_IN_WHITELIST       on
# shortcircuit USER_IN_DEF_WHITELIST   on
# shortcircuit USER_IN_ALL_SPAM_TO     on
# shortcircuit SUBJECT_IN_WHITELIST    on

#   the opposite; blacklisted mails can also save CPU
#
# shortcircuit USER_IN_BLACKLIST       on
# shortcircuit USER_IN_BLACKLIST_TO    on
# shortcircuit SUBJECT_IN_BLACKLIST    on

#   if you have taken the time to correctly specify your "trusted_networks",
#   this is another good way to save CPU
#
# shortcircuit ALL_TRUSTED             on

#   and a well-trained bayes DB can save running rules, too
#
# shortcircuit BAYES_99                spam
# shortcircuit BAYES_00                ham

endif # Mail::SpamAssassin::Plugin::Shortcircuit

Sorry for another long post again. :oops:

User avatar
mattg
Moderator
Moderator
Posts: 20294
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: SpamAssassin Questions

Post by mattg » 2018-02-05 12:31

thomas10 wrote:But how about DNSBL and SURBL entries?
This is my list

Code: Select all

DNSBL ENTRIES:
                    bl.spamcop.net      Score: 4     Result: 127.0.0.2
                  zen.spamhaus.org      Score: 4     Result: 127.0.0.2|127.0.0.4-8|127.0.0.10-11
            b.barracudacentral.org      Score: 4     Result: 127.0.0.2|127.0.0.4
                   cbl.abuseat.org      Score: 4     Result: 127.0.0.2
              aspews.ext.sorbs.net      Score: 4     Result: 127.0.0.2
                   dnsbl.sorbs.net      Score: 2     Result: 127.0.0.2|127.0.0.3|127.0.0.4|127.0.0.5|127.0.0.6|127.0.0.7|127.0.0.8|127.0.0.9|127.0.0.10|127.0.0.11|127.0.0.12|127.0.0.14
                ubl.unsubscore.com      Score: 1     Result: 127.0.0.2
                  sbl.spamhaus.org      Score: 10    Result: 127.0.0.3
                  sbl.spamhaus.org      Score: 6     Result: 127.0.0.9
              zz.countries.nerd.dk      Score: 1     Result: 127.0.0.1-35|127.0.0.37-255|127.0.1.*|127.0.2.0-41|127.0.2.43-255|127.0.3.1-71|127.0.3.73-255
     hostkarma.junkemailfilter.com      Score: 1     Result: 127.0.0.2|127.0.0.4
               all.bl.blocklist.de      Score: 1     Result: 127.0.0.2-22
                  all.spamrats.com      Score: 1     Result: 127.0.0.38|127.0.0.43
               ix.dnsbl.manitu.net      Score: 1     Result: 127.0.0.1

SURBL ENTRIES:
                   multi.surbl.org      Score: 3
                  dbl.spamhaus.org      Score: 3
        uribl.spameatingmonkey.net      Score: 1
                   uribl.swinog.ch      Score: 1
Because of this and intense scripts including pauses, SpamAssassin via a network (not local PC),etc, on my system Spam checking takes up to 30 seconds

Also remember that I am in Australia, and I score all non-Australian IP addresses
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-06 04:00

mattg wrote:
thomas10 wrote:But how about DNSBL and SURBL entries?
This is my list

Code: Select all

DNSBL ENTRIES:
                    bl.spamcop.net      Score: 4     Result: 127.0.0.2
                  zen.spamhaus.org      Score: 4     Result: 127.0.0.2|127.0.0.4-8|127.0.0.10-11
            b.barracudacentral.org      Score: 4     Result: 127.0.0.2|127.0.0.4
                   cbl.abuseat.org      Score: 4     Result: 127.0.0.2
              aspews.ext.sorbs.net      Score: 4     Result: 127.0.0.2
                   dnsbl.sorbs.net      Score: 2     Result: 127.0.0.2|127.0.0.3|127.0.0.4|127.0.0.5|127.0.0.6|127.0.0.7|127.0.0.8|127.0.0.9|127.0.0.10|127.0.0.11|127.0.0.12|127.0.0.14
                ubl.unsubscore.com      Score: 1     Result: 127.0.0.2
                  sbl.spamhaus.org      Score: 10    Result: 127.0.0.3
                  sbl.spamhaus.org      Score: 6     Result: 127.0.0.9
              zz.countries.nerd.dk      Score: 1     Result: 127.0.0.1-35|127.0.0.37-255|127.0.1.*|127.0.2.0-41|127.0.2.43-255|127.0.3.1-71|127.0.3.73-255
     hostkarma.junkemailfilter.com      Score: 1     Result: 127.0.0.2|127.0.0.4
               all.bl.blocklist.de      Score: 1     Result: 127.0.0.2-22
                  all.spamrats.com      Score: 1     Result: 127.0.0.38|127.0.0.43
               ix.dnsbl.manitu.net      Score: 1     Result: 127.0.0.1

SURBL ENTRIES:
                   multi.surbl.org      Score: 3
                  dbl.spamhaus.org      Score: 3
        uribl.spameatingmonkey.net      Score: 1
                   uribl.swinog.ch      Score: 1
Because of this and intense scripts including pauses, SpamAssassin via a network (not local PC),etc, on my system Spam checking takes up to 30 seconds

Also remember that I am in Australia, and I score all non-Australian IP addresses

Oh wow, that's a long list of the entries. No offense, matt, but I just curious, if were to set the same entries as yours, will it cause high possibilty of FP since there might be false scoring and my set on delete threshold is 20?

User avatar
mattg
Moderator
Moderator
Posts: 20294
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: SpamAssassin Questions

Post by mattg » 2018-02-06 05:27

Absolutely

Please don't just copy what I do; please set your own score based on your own experience.

I have a pretty savage SpamAssassin setup, including a rule that just adds 2.2 to all message scores automatically, in SpamAssassin I mark as spam from -500 (yes that's negative 500), and use the SpamAssassin score in hmailserver. This means that I can whitelist in SpamAssassin, and it impacts the hMailserver score.

I use a custom script to first Autoban spammers, and then reject them.
Currently I autoban from 18, and reject from 15. My current record is a score of 184.
I have between 100 and 400 Autoban entries for spammers at any one time. These last 7 days.

I don't get many false positives - perhaps 2 a week, and I occasionally bounce some mail that I'd like to get (including a script in an attachment from jimimaseye in the UK last week)
I spend a lot of time checking logs, watching bounces, etc.

I also rescan unread mail every couple of hours, and move some of those to my spam account. I catch a few 'trojans as attachments' this way.
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-06 05:50

mattg wrote:Absolutely

Please don't just copy what I do; please set your own score based on your own experience.

I have a pretty savage SpamAssassin setup, including a rule that just adds 2.2 to all message scores automatically, in SpamAssassin I mark as spam from -500 (yes that's negative 500), and use the SpamAssassin score in hmailserver. This means that I can whitelist in SpamAssassin, and it impacts the hMailserver score.

I use a custom script to first Autoban spammers, and then reject them.
Currently I autoban from 18, and reject from 15. My current record is a score of 184.
I have between 100 and 400 Autoban entries for spammers at any one time. These last 7 days.

I don't get many false positives - perhaps 2 a week, and I occasionally bounce some mail that I'd like to get (including a script in an attachment from jimimaseye in the UK last week)
I spend a lot of time checking logs, watching bounces, etc.

I also rescan unread mail every couple of hours, and move some of those to my spam account. I catch a few 'trojans as attachments' this way.
Hmm, ok. In your experience, if I remain the entries as usual, do you think it is ok?
Below is my setting.

Code: Select all

-----------------------------------------------------------------------------------------------

ANTISPAM

GENERAL                              SPAM TESTS              Score   SPAMASSASSIN
  Spam Mark:                  5       Use SPF:            True - 3    Use Spamassassin:    True
  Add X-HmailServer-Spam:     True    Check HELO host:    True - 2    Hostname:       localhost
  Add X-HmailServer-Reason:   True    Check MX records:   True - 2    Port:                 783
  Add X-HmailServer-Subject:  True    Verify DKIM:       False        Use SA score: False -   5
              Subject Text: "[SPAM]"
  Spam delete threshold: 20         Maximum message size: 1024

DNSBL ENTRIES:
                  zen.spamhaus.org      Score: 3     Result: 127.0.0.2-8|127.0.0.10-11
                    bl.spamcop.net      Score: 3     Result: 127.0.0.2

SURBL ENTRIES:
                   multi.surbl.org      Score: 3

User avatar
mattg
Moderator
Moderator
Posts: 20294
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: SpamAssassin Questions

Post by mattg » 2018-02-06 07:42

That's OK

Do you also have ClamAV with SaneSecurity definitions? If not, I'd look into that

and I'd add 'snowshoe' to be higher score (like mine -thanks to SorenR for the lesson in what is snowshoe spam)
sbl.spamhaus.org Score: 10 Result: 127.0.0.3


you only give that a 3 with your zen.spamhaus.org setting
Everyone on that list is a confirmed scammer / spammer. >> https://www.spamhaus.org/faq/section/Glossary
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-06 08:23

mattg wrote:That's OK

Do you also have ClamAV with SaneSecurity definitions? If not, I'd look into that

and I'd add 'snowshoe' to be higher score (like mine -thanks to SorenR for the lesson in what is snowshoe spam)
sbl.spamhaus.org Score: 10 Result: 127.0.0.3


you only give that a 3 with your zen.spamhaus.org setting
Everyone on that list is a confirmed scammer / spammer. >> https://www.spamhaus.org/faq/section/Glossary

Ok matt, thanks for the info. My colleague has tried with clamwin, but ended up causing the email gone through very slow and the email queue became very long, so there is no antivirus linked with Hmail.

User avatar
jimimaseye
Moderator
Moderator
Posts: 8172
Joined: 2011-09-08 17:48

Re: SpamAssassin Questions

Post by jimimaseye » 2018-02-06 09:46

thomas10 wrote: Ok matt, thanks for the info. My colleague has tried with clamwin, but ended up causing the email gone through very slow and the email queue became very long, so there is no antivirus linked with Hmail.
I guarantee that he was using a non-converted CLAMWIN versions and not a service-based Clamd version. It is easily converted to stop this.

Take a read and decide for yourself: viewtopic.php?f=21&t=26829
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
jimimaseye
Moderator
Moderator
Posts: 8172
Joined: 2011-09-08 17:48

Re: SpamAssassin Questions

Post by jimimaseye » 2018-02-06 09:57

thomas10 wrote: Hmm, ok. In your experience, if I remain the entries as usual, do you think it is ok?
Below is my setting.

Code: Select all

-----------------------------------------------------------------------------------------------

ANTISPAM

GENERAL                              SPAM TESTS              Score   SPAMASSASSIN
  Spam Mark:                  5       Use SPF:            True - 3    Use Spamassassin:    True
  Add X-HmailServer-Spam:     True    Check HELO host:    True - 2    Hostname:       localhost
  Add X-HmailServer-Reason:   True    Check MX records:   True - 2    Port:                 783
  Add X-HmailServer-Subject:  True    Verify DKIM:       False        Use SA score: False -   5
              Subject Text: "[SPAM]"
  Spam delete threshold: 20         Maximum message size: 1024

DNSBL ENTRIES:
                  zen.spamhaus.org      Score: 3     Result: 127.0.0.2-8|127.0.0.10-11
                    bl.spamcop.net      Score: 3     Result: 127.0.0.2

SURBL ENTRIES:
                   multi.surbl.org      Score: 3
Spam delete threshold of 20 is very high and nothing will get deleted (all your scoring will never hit 20: 5+3x3 = 14). I recommend you change it to 8.

Spamassassin (99% will mark genuine spam) = 5.
spamhaus = 3
TOTAL = 8 (if those to have marked then it is definitely spam IMO).

MY settings :
[code]
ANTISPAM

GENERAL SPAM TESTS Score SPAMASSASSIN
Spam Mark: 5 Use SPF: True - 3 Use Spamassassin: True
Add X-HmailServer-Spam: True Check HELO host: True - 2 Hostname: 127.0.0.1
Add X-HmailServer-Reason: True Check MX records: True - 2 Port: 783
Add X-HmailServer-Subject: True Verify DKIM: False - 5 Use SA score: False - 5
Subject Text: "[SPAM]"
Spam delete threshold: 8 Maximum message size: 2048

GREYLISTING:
Greylisting: False

DNSBL ENTRIES:
zen.spamhaus.org Score: 5 Result: 127.0.0.2-8|127.0.0.10-11
bl.spamcop.net Score: 3 Result: 127.0.0.2
b.barracudacentral.org Score: 2 Result: 127.0.0.2
hostkarma.junkemailfilter.com Score: 2 Result: 127.0.0.2|127.0.0.4
bl.spameatingmonkey.net Score: 2 Result: 127.0.0.2-3
cbl.abuseat.org Score: 2 Result: 127.0.0.2

SURBL ENTRIES:
multi.surbl.org Score: 3
-----------------------------------------------------------------------------------------------[/code]

(I have spamhaus as 5 and the handful of extra DNSBL entries help to prtect in the event spamassassin doesnt catch for some reason without there being too many causing an over-exageration of FP's (if you know what I mean)).
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-06 10:14

jimimaseye wrote:
thomas10 wrote: Hmm, ok. In your experience, if I remain the entries as usual, do you think it is ok?
Below is my setting.

Code: Select all

-----------------------------------------------------------------------------------------------

ANTISPAM

GENERAL                              SPAM TESTS              Score   SPAMASSASSIN
  Spam Mark:                  5       Use SPF:            True - 3    Use Spamassassin:    True
  Add X-HmailServer-Spam:     True    Check HELO host:    True - 2    Hostname:       localhost
  Add X-HmailServer-Reason:   True    Check MX records:   True - 2    Port:                 783
  Add X-HmailServer-Subject:  True    Verify DKIM:       False        Use SA score: False -   5
              Subject Text: "[SPAM]"
  Spam delete threshold: 20         Maximum message size: 1024

DNSBL ENTRIES:
                  zen.spamhaus.org      Score: 3     Result: 127.0.0.2-8|127.0.0.10-11
                    bl.spamcop.net      Score: 3     Result: 127.0.0.2

SURBL ENTRIES:
                   multi.surbl.org      Score: 3
Spam delete threshold of 20 is very high and nothing will get deleted (all your scoring will never hit 20: 5+3x3 = 14). I recommend you change it to 8.

Spamassassin (99% will mark genuine spam) = 5.
spamhaus = 3
TOTAL = 8 (if those to have marked then it is definitely spam IMO).

MY settings :
[code]
ANTISPAM

GENERAL SPAM TESTS Score SPAMASSASSIN
Spam Mark: 5 Use SPF: True - 3 Use Spamassassin: True
Add X-HmailServer-Spam: True Check HELO host: True - 2 Hostname: 127.0.0.1
Add X-HmailServer-Reason: True Check MX records: True - 2 Port: 783
Add X-HmailServer-Subject: True Verify DKIM: False - 5 Use SA score: False - 5
Subject Text: "[SPAM]"
Spam delete threshold: 8 Maximum message size: 2048

GREYLISTING:
Greylisting: False

DNSBL ENTRIES:
zen.spamhaus.org Score: 5 Result: 127.0.0.2-8|127.0.0.10-11
bl.spamcop.net Score: 3 Result: 127.0.0.2
b.barracudacentral.org Score: 2 Result: 127.0.0.2
hostkarma.junkemailfilter.com Score: 2 Result: 127.0.0.2|127.0.0.4
bl.spameatingmonkey.net Score: 2 Result: 127.0.0.2-3
cbl.abuseat.org Score: 2 Result: 127.0.0.2

SURBL ENTRIES:
multi.surbl.org Score: 3
-----------------------------------------------------------------------------------------------[/code]

(I have spamhaus as 5 and the handful of extra DNSBL entries help to prtect in the event spamassassin doesnt catch for some reason without there being too many causing an over-exageration of FP's (if you know what I mean)).
Lovely, guess I will change the delete threshold and spamhaus then see how it goes. Will think of add more entries if needed. As for clamwin, I will look through the link you sent. Thanks so much.

BTW, in future, If I need to move Hmail to other PC, how to migrate the SA?

User avatar
jimimaseye
Moderator
Moderator
Posts: 8172
Joined: 2011-09-08 17:48

Re: SpamAssassin Questions

Post by jimimaseye » 2018-02-06 14:46

If you migrate to a new box, simply install Spamassassin as you did the first time round and copy over any custom .CF files (such as local.cf). if you have been using BAYES training you may wish to retrain the bayes database against (using sa-learn) or you could simply copy ov the existing bayes database from your current system. Those databases are found somewhere amongst:

C:\Windows\SysWOW64\config\systemprofile\.spamassassin\bayes
C:\Documents and Settings\Default User\.spamassassin\bayes
C:\Users\Administrator\.spamassassin\bayes
(these are the default locations, dependant on what OS you are running) or wherever your --dbpath variable in your spamassassin config file points to if you have one.

Whether you move the Bayes database across or not or simply start it from scratch again is spoething only you can decide on as it will be based on your reliance and throughput of it. (Of course, you may not be using Bayes in which case the above is all irrelevant).

alternatively, you may not wish to move spamassassin to the other box (with the new Hmailserver) - just simply point the new Hmailserver installation to the existing box (ensure you have port 783 open through the firewalls).
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-07 08:47

Ok, will keep this as record. Will try to do proper SA setup and get it ready soon. Hope things go fine. Will post again if got any question.
Cheers guys, thank you all so much, appreciate that. :mrgreen:

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-07 11:10

jimimaseye wrote: Spam delete threshold of 20 is very high and nothing will get deleted (all your scoring will never hit 20: 5+3x3 = 14). I recommend you change it to 8.
I am abit confused on the Delete threshold calculation, how does it being calculated?
I thought it is as below:
5 (Spam Mark)+ 3 (SPF)+ 2 (HELO)+ 2 (MX)+ 5 (SA)+ (3x3)[Entries)= 26

User avatar
jimimaseye
Moderator
Moderator
Posts: 8172
Joined: 2011-09-08 17:48

Re: SpamAssassin Questions

Post by jimimaseye » 2018-02-07 11:24

thomas10 wrote:5 (Spam Mark)+ 3 (SPF)+ 2 (HELO)+ 2 (MX)+ 5 (SA)+ (3x3)[Entries)= 26
close - but the SPAM MARK (5) is not a scoring, but a threshold. As in when the total score reashes 5, it will be MARKED as SPAM.

So to correct your calculation:
thomas10 wrote:5 (Spam Mark)+ 3 (SPF)+ 2 (HELO)+ 2 (MX)+ 5 (SA)+ (3x3)[Entries)= 21
but of course the individual scores only apply if there is a hit under that particular check.

Trust me.... set it to 8.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-07 11:28

jimimaseye wrote:
thomas10 wrote:5 (Spam Mark)+ 3 (SPF)+ 2 (HELO)+ 2 (MX)+ 5 (SA)+ (3x3)[Entries)= 26
close - but the SPAM MARK (5) is not a scoring, but a threshold. As in when the total score reashes 5, it will be MARKED as SPAM.

So to correct your calculation:
thomas10 wrote:5 (Spam Mark)+ 3 (SPF)+ 2 (HELO)+ 2 (MX)+ 5 (SA)+ (3x3)[Entries)= 21
but of course the individual scores only apply if there is a hit under that particular check.

Trust me.... set it to 8.

LoL, I have forgotten that spam mark is for [SPAM] marking. Thanks for explaining.
ok, will set it as 8 and find a day to move Hmail to this PC and test it. Thanks so much. :D

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-08 04:09

Sorry to ask again. How about whitelist/ blacklist?
If to set whitelist/blacklist in local.cf, it only affect for SA only? Please correct me if I am wrong in this.

And curious to know, based on your experience, will false positve email score high or it normally in the [Mark Spam] range?

User avatar
jimimaseye
Moderator
Moderator
Posts: 8172
Joined: 2011-09-08 17:48

Re: SpamAssassin Questions

Post by jimimaseye » 2018-02-08 09:46

If to set whitelist/blacklist in local.cf, it only affect for SA only?
Correct.
And curious to know, based on your experience, will false positve email score high or it normally in the [Mark Spam] range?
I dont remeber seeing a false positive that was deleted due to being so high. Most FP's would simply be marked spam....but then I rarely get any FP's anyway. (Less than 10 in all last year). If anything, NEW spam will get through without being marked spam (leaving it to user diligence to notice and delete). But this is normal especially when a new batch/flood of spam is released and its Zero hour.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-08 09:58

jimimaseye wrote:
I dont remeber seeing a false positive that was deleted due to being so high. Most FP's would simply be marked spam....but then I rarely get any FP's anyway. (Less than 10 in all last year). If anything, NEW spam will get through without being marked spam (leaving it to user diligence to notice and delete). But this is normal especially when a new batch/flood of spam is released and its Zero hour.
Ok, thanks for your answer. The hmail here is processing ten of thousands email per day, that's why I am concerned on FP because will face trouble if genuine email being deleted due to FP.
Plan to start it next week. Hope everything will be ok. Once stable, perhaps ClamAV is next. :wink:

User avatar
jimimaseye
Moderator
Moderator
Posts: 8172
Joined: 2011-09-08 17:48

Re: SpamAssassin Questions

Post by jimimaseye » 2018-02-08 10:04

ALL of my FP's I receive are due to my personaly rules and not triggered by default spamasssassin rules. (I have a rule that has something along the lines of "Dear [non-personal name. eg, Sir/Madam/Customer/User] and the body containing words like "account" or "verify". It a lot more complicated than that but that is the essence of it).
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-08 10:36

jimimaseye wrote:ALL of my FP's I receive are due to my personaly rules and not triggered by default spamasssassin rules. (I have a rule that has something along the lines of "Dear [non-personal name. eg, Sir/Madam/Customer/User] and the body containing words like "account" or "verify". It a lot more complicated than that but that is the essence of it).
Ya, the Non-personal name. That's one of the popular term in spam mail. Ok, thanks for your info on FP. Really appreciated.

How about the the custom rules from Kopis.com? ( viewtopic.php?f=21&t=28133#p179267 )
I have downloaded the zip file from your post, the only thing I need to do is saving the content into C:\Program Files (x86)\JAM Software\SpamAssassin for Windows\share\? The folder name a bit different due to the SA version.

After saving it, I need to only restart SA service or Hmail service together? :roll:

User avatar
jimimaseye
Moderator
Moderator
Posts: 8172
Joined: 2011-09-08 17:48

Re: SpamAssassin Questions

Post by jimimaseye » 2018-02-08 11:21

Restart the spamassassin service.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-08 11:50

jimimaseye wrote:Restart the spamassassin service.
Well noted. Thanks jimi and matt for the advices. :mrgreen:

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-13 09:10

Guys, If were to train bayes, will SA continue run in Hmailserver in the same time when Bayes training is ongoing?

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-14 12:38

I have read documentation and there got mention about language option. Is SpamAssassin supported to score for emails in languages other than English (Chinese, Thailand, Cambodia, Myanmar language)? If there is a need to make changes in cf file, what should I do?

User avatar
mattg
Moderator
Moderator
Posts: 20294
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: SpamAssassin Questions

Post by mattg » 2018-02-15 01:45

https://spamassassin.apache.org/full/3. ... xtCat.html

Here's mine

Code: Select all

ok_locales en

ifplugin Mail::SpamAssassin::Plugin::TextCat

# This option is used to specify which languages are considered OK for incoming mail.
# SpamAssassin will try to detect the language used in the message text.

ok_languages nl en

inactive_languages bs cy eo et eu fy ga gd is la lt lv rm sa sco sl yi fr de

endif # Mail::SpamAssassin::Plugin::TextCat
You need to enable the TextCat plugin, and you can score form these as well using UNWANTED_LANGUAGE_BODY
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-21 11:12

Ok thanks matt.

I am abit confused on this part. I tend get it clear before set the correct global rules
1) For X-hMailServer-Reason-Score, is it only add the total score of other things except SA score? or it totaled up the whole things?
2) X-Spam-Level is for SA score only?
Below is my current setting.

Code: Select all

ANTISPAM										
GENERAL                              SPAM TESTS              Score   SPAMASSASSIN		
  Spam Mark:                  5       Use SPF:            True - 3    Use Spamassassin:    True
  Add X-HmailServer-Spam:     True    Check HELO host:    True - 2    Hostname:       localhost
  Add X-HmailServer-Reason:   True    Check MX records:   True - 2    Port:                 783
  Add X-HmailServer-Subject:  True    Verify DKIM:       False - 5    Use SA score: False -   5
              Subject Text: "[SPAM]"						
  Spam delete threshold: 8         Maximum message size: 1024				

DNSBL ENTRIES:									
                  zen.spamhaus.org      Score: 3     Result: 127.0.0.2-8|127.0.0.10-11
                    bl.spamcop.net      Score: 3     Result: 127.0.0.2		
SURBL ENTRIES:									
                   multi.surbl.org      Score: 3				

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-21 12:09

Ok, for global rules, I think it sounds stupid.
From the beginning, I need to forward all Spam marked email to other spam collecting email. So the rules should be:

Criteria -> Subject -> Equals: '[SPAM]'
Action -> Forward to: 'Spam collecting Mail'

But once confirm that emails score more than 8 are mostly spam, I need to change the global rule.
Condition: Score above 5 but below 8, action : forward.

Then back to my previous post on X-hMailServer-Reason-Score and X-Spam-Level. I need to know which should I choose. Thanks for assisting again guys.

User avatar
jimimaseye
Moderator
Moderator
Posts: 8172
Joined: 2011-09-08 17:48

Re: SpamAssassin Questions

Post by jimimaseye » 2018-02-21 15:06

If you read the Spamassassin installation thread (where I posted my solution) it expains specifically why the global rules are required. In particular, they use the SA scoring and the marking by the asterix instad of looking for the word "[SPAM]" in the subject for a reason. Consider this:

1, You send an email to someone.

2, They have an over-zealous antispam system that marks your legitimate email as spam but the user knows its an FP and replies to you anyway.

3, You now receive a legitimate reply that has the word [SPAM] in the subject - it gets put to spam folder/bin (and youur system didnt even think it was spam in the first place).

However, if you use the spamassassin header and asterix counter then you will know what the score is and that YOUR system determined it to be so or not.


NOTE: if you have a rule that FORWARDS to an address and that address is also on your system then it will loop around forwarding itself unless you check the x-hamailserver-loopcount header and factor that in your rule.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-22 03:15

jimimaseye wrote:If you read the Spamassassin installation thread (where I posted my solution) it expains specifically why the global rules are required. In particular, they use the SA scoring and the marking by the asterix instad of looking for the word "[SPAM]" in the subject for a reason. Consider this:

1, You send an email to someone.

2, They have an over-zealous antispam system that marks your legitimate email as spam but the user knows its an FP and replies to you anyway.

3, You now receive a legitimate reply that has the word [SPAM] in the subject - it gets put to spam folder/bin (and youur system didnt even think it was spam in the first place).

However, if you use the spamassassin header and asterix counter then you will know what the score is and that YOUR system determined it to be so or not.


NOTE: if you have a rule that FORWARDS to an address and that address is also on your system then it will loop around forwarding itself unless you check the x-hamailserver-loopcount header and factor that in your rule.
So the rules should be:
Criteria -> Custom header field: 'X-Spam-Level' Contains '********' << 8 asterix
Criteria -> Custom header field: 'X-hMailServer-LoopCount' Less than: '1'
Action -> Forward to email

I put 8 asterix as I assume I can get all the email that score more than 8 to prevent FP.
Also with the loopcount <1.
Just to confirm again whether the rule above ok as my condition is to forward emails score more than 8 but less than 5 to my email?

User avatar
mattg
Moderator
Moderator
Posts: 20294
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: SpamAssassin Questions

Post by mattg » 2018-02-22 04:07

thomas10 wrote:1) For X-hMailServer-Reason-Score, is it only add the total score of other things except SA score? or it totaled up the whole things?
Total of all
thomas10 wrote:2) X-Spam-Level is for SA score only?
Yes, that's a SpamAssasin Header
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-22 04:46

mattg wrote:
thomas10 wrote:1) For X-hMailServer-Reason-Score, is it only add the total score of other things except SA score? or it totaled up the whole things?
Total of all
thomas10 wrote:2) X-Spam-Level is for SA score only?
Yes, that's a SpamAssasin Header
Hi Matt, Sorry for back and forth on the rules setting.
How about this?
Forward and Delete email that score more than 8 (Delete Threshold:8)

Code: Select all

Criteria -> Custom header field: 'X-hMailServer-Spam' Equals: 'YES' 
AND
Criteria -> Custom header field: 'X-hMailServer-Reason-Score' Greater than: '8'
AND
Criteria -> Custom header field: 'Delivered-To' Not contains: ' Spam collecting Email'
AND
Criteria -> Custom header field: 'X-hMailServer-LoopCount' Less than: '1'
Action -> Forward: spam_mail@sin.global-gp.com
Action -> Delete
Forward Email that score less than 8 but more than 5 (Mark Spam Threshold: 5)

Code: Select all

Criteria -> Custom header field: 'X-hMailServer-Spam' Equals: 'YES' 
Criteria -> Custom header field: 'X-hMailServer-Reason-Score' less than: '8'
Criteria -> Custom header field: 'X-hMailServer-Reason-Score' Greater than: '5'
Criteria -> Custom header field: 'Delivered-To' Not contains: 'spam_mail@sin.global-gp.com’
Criteria -> Custom header field: 'X-hMailServer-LoopCount' Less than: '1'
Action -> Forward: 'Spam collecting Email'

User avatar
mattg
Moderator
Moderator
Posts: 20294
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: SpamAssassin Questions

Post by mattg » 2018-02-22 05:03

While you have a delete threshold of 8, these will just be rejected, and not forwarded to the SPAM Account.

like this

incoming server says 'HELO/EHLO'
Your server say 'EHLO'
Incoming server says I have mail for you@example.com
Your server says OK
Incoming server says if have mail from abc@spammer.com
your server says OK, please send it
Incoming server sends message
Your server does spam checks, including SpamAssassin
Your server works out this is likely spam (score higher than 8 in your case)
Your server says 'rejected, I think this is spam, and here's why' lists last spam test that enabled spam score to reach higher than 8

If you want to inspect this mail, you need to set your spam delete threshold higher, so it actually gets to the global rules

ALSO, a delete in the rules, as you have done, will simply stop the mail being delivered to the original recipient.
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-22 05:15

Ok matt, thanks for your explanation. So there is no need to set rules to forward email that score more than 8 as it is already deleted due to Delete Threshold:8.

To receive higher score email, Set the Delete Threshold higher.

Lastly, a delete set in the rules, email won't go to the original recipients, but can forward to email that being set.

Well noted, thanks matt and jimi for this SA setting journey. :D

User avatar
mattg
Moderator
Moderator
Posts: 20294
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: SpamAssassin Questions

Post by mattg » 2018-02-22 07:02

If you set delete threshold at 8, but use a set score for SpamAssassin as jimimaseye does, then using a rule that catches
Custom header field: 'X-Spam-Level' Contains '********' << 8 asterix

will catch mail that SpamAssassin sees as higher than 8 on it's own.
It is possible that all other hmailserver tests pass and then a SPamassassin score of 15 would only add (say) 4 to the hMailserver Spam Score, then you would have a score of 4 in hMailserver.
This would miss your normal 'mark as spam' score of 4, and so may go through to the intended recipient.

There are many, many ways to set spam scores, and every change that you make will change other things.
I've just started External POP3 downloads for a few accounts, and these have spam scores that are different because my normal SpamAssassin SPF rules fail these messages. I'm still working on a better solution than I currently have...Always fine tuning
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-23 09:37

mattg wrote:If you set delete threshold at 8, but use a set score for SpamAssassin as jimimaseye does, then using a rule that catches
Custom header field: 'X-Spam-Level' Contains '********' << 8 asterix

will catch mail that SpamAssassin sees as higher than 8 on it's own.
It is possible that all other hmailserver tests pass and then a SPamassassin score of 15 would only add (say) 4 to the hMailserver Spam Score, then you would have a score of 4 in hMailserver.
This would miss your normal 'mark as spam' score of 4, and so may go through to the intended recipient.

There are many, many ways to set spam scores, and every change that you make will change other things.
I've just started External POP3 downloads for a few accounts, and these have spam scores that are different because my normal SpamAssassin SPF rules fail these messages. I'm still working on a better solution than I currently have...Always fine tuning
matt, if to set with 8 asterix, it will only catch email scores higher than 8? how about below 8?
if to set that, loop count < 1 rule also needs to set?

Understood on your explanation, since I set untick 'use SA score' with score:5, even if SA scores the email higher than 5, the score passed to hmail for calculation is still only 5 (Haven't added with other spam score). Correct me if I am wrong.

In your experience, X-Spam-Level vs X-hMailServer-Reason-Score, which is better to be searched by global rule?

User avatar
jimimaseye
Moderator
Moderator
Posts: 8172
Joined: 2011-09-08 17:48

Re: SpamAssassin Questions

Post by jimimaseye » 2018-02-23 09:59

There are 2 scoring systems (using my method) and its quite simple to configure: you can respond to the Spamassassin scoring OR/AND you can respond to the HMS scoring system.

If you want to react (by a rule) only to SA scoring then count the asterix ('X-Spam-Level' Contains '********')
If you want to react to HMS system then review the header X-hMailServer-Spam' Equals: 'YES' or X-hMailServer-Reason-Score. (HMS scoring system is also used by the 'Spam Mark' and 'Spam Delete' thresholds - sort of like a built in rule)

Remember that an SA score of at least 3 (ie 3 asterix) will acquire a HMS score of at least 5 (hence the mark spam threshold at 5)
A final HMS score of 8 will delete the email.

With that you should be able to work out what to do. If you want full control and movement of your emails (based on the scoring) by RULES only then simply raise the Spam Delete threshold to something unfeasible so nothing gets deleted.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-23 10:51

jimimaseye wrote:
If you want to react (by a rule) only to SA scoring then count the asterix ('X-Spam-Level' Contains '********')
If you want to react to HMS system then review the header X-hMailServer-Spam' Equals: 'YES' or X-hMailServer-Reason-Score. (HMS scoring system is also used by the 'Spam Mark' and 'Spam Delete' thresholds - sort of like a built in rule)

Remember that an SA score of at least 3 (ie 3 asterix) will acquire a HMS score of at least 5 (hence the mark spam threshold at 5)
A final HMS score of 8 will delete the email.
Ah jimi, Understood that. Back to the scoring, for SA setting in Hmail, it was untick to not using SA score with 5(same as yours). Does it mean the score will be capped as 5 and won't go further higher number? or it is just like the local cf require score:5 to have the email marked? I was confused on it.

With that you should be able to work out what to do. If you want full control and movement of your emails (based on the scoring) by RULES only then simply raise the Spam Delete threshold to something unfeasible so nothing gets deleted.
I was thinking to have email that scores more than 5 to be forwarded to spam catch account and delete email that scores 8 and above, so delete threshold was set.

Understand from matt that once score 8 or higher, it will be rejected and won't be able to forward to spam catch account. Thus, rules for 'X-hMailServer-Reason-Score' Greater than 8 to forward seems not working.

So I was finding whether if use X-Spam-Level rules, will it able to catch the email scored higher than 8 before it get deleted? Just wonder on it. If the answer is No, then I will be happy too because I have a confirm answer.

Jimi, Do you set any rule on languages in local cf file? (like ok_locales, ok_languages) Because I was planning to insert Chinese rules cf file so have it worked, but unsure whether I need to change some setting or not.
Appreciated you two for the excellent Q&A. Thanks again.

User avatar
mattg
Moderator
Moderator
Posts: 20294
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: SpamAssassin Questions

Post by mattg » 2018-02-23 16:35

thomas10 wrote:Back to the scoring, for SA setting in Hmail, it was untick to not using SA score with 5(same as yours). Does it mean the score will be capped as 5 and won't go further higher number? or it is just like the local cf require score:5 to have the email marked? I was confused on it.
SpamAssassin score independently from hMailserver.

If your SpamAssassin 'required_score' was say 3, and then scored at least that number, then it will say that the message is SPAM

If in hMailsevrer you set to NOT use SPamAssassin score, but instead choose 5 as the SpamAssassin score, then no matter the spamAssassin score, if 3 or more, hmailserver will score as 5
For instance a message gets a 3.1 in SpamAssassin - hmailserver says 5
If a message gets 150 in SpamAssassin, then hmailserver still says 5

If SpamAssassin scores 2.9 (with a required_score of 3.0 being set), then the spamassassin will 'pass' the message, and hMailserver scores 0
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-26 06:38

mattg wrote:
2018-02-23 16:35
SpamAssassin score independently from hMailserver.

If your SpamAssassin 'required_score' was say 3, and then scored at least that number, then it will say that the message is SPAM

If in hMailsevrer you set to NOT use SPamAssassin score, but instead choose 5 as the SpamAssassin score, then no matter the spamAssassin score, if 3 or more, hmailserver will score as 5
For instance a message gets a 3.1 in SpamAssassin - hmailserver says 5
If a message gets 150 in SpamAssassin, then hmailserver still says 5

If SpamAssassin scores 2.9 (with a required_score of 3.0 being set), then the spamassassin will 'pass' the message, and hMailserver scores 0
Lovely, matt, your explanation on the scoring is truly understandable, really appreciate it.

User avatar
jimimaseye
Moderator
Moderator
Posts: 8172
Joined: 2011-09-08 17:48

Re: SpamAssassin Questions

Post by jimimaseye » 2018-02-26 09:38

thomas10 wrote:
mattg wrote:
2018-02-23 16:35
If your SpamAssassin 'required_score' was say 3, and then scored at least that number, then it will say that the message is SPAM
Actually it will mark according to whatever you set in your local.cf under rewrite_header Subject. In my setup it doesnt write "spam"- I have it put the score ("[_hits_]").

(I only put the word "[SPAM]" in when it hits the HMS threshold.)
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-26 10:08

jimimaseye wrote:
2018-02-26 09:38
thomas10 wrote:
mattg wrote:
2018-02-23 16:35
If your SpamAssassin 'required_score' was say 3, and then scored at least that number, then it will say that the message is SPAM
Actually it will mark according to whatever you set in your local.cf under rewrite_header Subject. In my setup it doesnt write "spam"- I have it put the score ("[_hits_]").

(I only put the word "[SPAM]" in when it hits the HMS threshold.)
Yup, Same as yours m using HITS and [SPAM] in hmail. only that the required_score on my side is 5. :D

User avatar
jimimaseye
Moderator
Moderator
Posts: 8172
Joined: 2011-09-08 17:48

Re: SpamAssassin Questions

Post by jimimaseye » 2018-02-26 10:17

thomas10 wrote:
2018-02-26 10:08
Yup, Same as yours m using HITS and [SPAM] in hmail. only that the required_score on my side is 5. :D
I find a threshold of 5 is too high leaving many spam emails free entry without being marked. I never see valid emails over 2 (max) and usually below zero.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-26 10:46

jimimaseye wrote:
2018-02-26 10:17
thomas10 wrote:
2018-02-26 10:08
Yup, Same as yours m using HITS and [SPAM] in hmail. only that the required_score on my side is 5. :D
I find a threshold of 5 is too high leaving many spam emails free entry without being marked. I never see valid emails over 2 (max) and usually below zero.
No worries jimi, I set this just to see how many genuine spam mail escaped from it. I will change it to 3 once ok.
BTW, I choose to set global rules using X-hMailServer-Spam: YES and X-hMailServer-Reason-Score Greater than 5 and less than 8 to check the email marked as Spam first. If found it not good enough then I will go for X-Spam. Thanks for your guidance post also. Thanks again.

Just move hmail to another server, now I am having problem to migrate the user email data over using Data Directory Synchronizer (DDS). When run it, after few minutes, it seems to slow down the email server because my outlook receives message of Pop3 timed out. If I close it, it can't continue where it stops by and need to start from beginning, is there any way?

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-26 11:05

Problem when trying to test the SA.
it said unable to connect to the specific SA server. Why?

User avatar
jimimaseye
Moderator
Moderator
Posts: 8172
Joined: 2011-09-08 17:48

Re: SpamAssassin Questions

Post by jimimaseye » 2018-02-26 11:10

thomas10 wrote:
2018-02-26 10:46
Just move hmail to another server, now I am having problem to migrate the user email data over using Data Directory Synchronizer (DDS). When run it, after few minutes, it seems to slow down the email server because my outlook receives message of Pop3 timed out. If I close it, it can't continue where it stops by and need to start from beginning, is there any way?
Oooh. using DDS is not so good for migration because ALL emails will be put in the INBOX of the account including sent emails. Why didnt you do a standard backp/restore?

For DDS speed: viewtopic.php?f=21&t=29828
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-26 11:24

Update:
1) While clicked Test, it mentioned "unable to connect to the specific SA server."
2) The spamd.log isn't created.

User avatar
jimimaseye
Moderator
Moderator
Posts: 8172
Joined: 2011-09-08 17:48

Re: SpamAssassin Questions

Post by jimimaseye » 2018-02-26 11:55

Therefore the Service is not running or responding.

Check the service is installed correctly
Check that the address/port is set correctly in HMS
Check the service is running
Check the firewall allows the comms (port 783)
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-26 11:59

jimimaseye wrote:
2018-02-26 11:55
Therefore the Service is not running or responding.

Check the service is installed correctly
Check that the address/port is set correctly in HMS
Check the service is running
Check the firewall allows the comms (port 783)
Hohoho, I have no idea where I did wrong,
Service install correctly and started.
localhost and port 783 is correct
service running
firewall allowed

Updated: Found that I need to double click the spamd.exe to have it run and then the test worked. How in the hell it happens?lol

User avatar
jimimaseye
Moderator
Moderator
Posts: 8172
Joined: 2011-09-08 17:48

Re: SpamAssassin Questions

Post by jimimaseye » 2018-02-26 12:06

thomas10 wrote:
2018-02-26 11:59

Updated: Found that I need to double click the spamd.exe to have it run and then the test worked. How in the hell it happens?lol
That means that you dont have the SERVICE running then. Manually running the spamd.exe is not running the service. Set the spamd to run as a service so it starts as a process when the box starts and you dont need to manually run it.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-26 12:21

jimimaseye wrote:
2018-02-26 12:06
thomas10 wrote:
2018-02-26 11:59

Updated: Found that I need to double click the spamd.exe to have it run and then the test worked. How in the hell it happens?lol
That means that you dont have the SERVICE running then. Manually running the spamd.exe is not running the service. Set the spamd to run as a service so it starts as a process when the box starts and you dont need to manually run it.
Weird. I have delete the service and recreate it, but doesn't seem to work. The problem still same.
I do steps below:
1. cmd for "C:\Program Files\Windows Resource Kits\Tools\INSTSRV.EXE" SpamAssassin "C:\Program Files\Windows Resource Kits\Tools\SRVANY.EXE" to create SpamAssassin Service.
2. Regedit-> HKLM -> SYSTEM -> CurrentControlSet -> Services -> SpamAssassin
a. Add a new key called under SpamAssassin of "Parameters"
b. In Parameters, add a new String Value of Application with value to
"C:\Program Files\JAM Software\SpamAssassin for Windows\spamd.exe" -l -s "C:\Program Files (x86)\hMailServer\Logs\spamd.log" --max-spare=2 --max- children=20 --timeout-child=20
c. Create a second String value in Parameters of AppDirectory with value below:
"C:\Program Files\JAM Software\SpamAssassin for Windows
3. Start SA service
4. hmailserver / Settings / Anti Spam / SpamAssassin , enable and save.

Found the error log regarding SA.
"ERROR" 1352 "2018-02-26 18:19:04.352" "Severity: 2 (High), Code: HM5508,
Source: SpamAssassinTestConnect::TestConnect, Description: The SpamAssassin
tests did not complete. Please confirm that the configuration (host name and
port) is valid and that SpamAssassin is running."
I tried hostname either localhost or 127.0.0.1 but still not working.

User avatar
jimimaseye
Moderator
Moderator
Posts: 8172
Joined: 2011-09-08 17:48

Re: SpamAssassin Questions

Post by jimimaseye » 2018-02-26 12:31

I cant help you with that type of installation of the service as I dont do it that way.

I do it a little simpler: viewtopic.php?f=21&t=28133 (using NSSM)
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-26 12:58

jimimaseye wrote:
2018-02-26 12:31
I cant help you with that type of installation of the service as I dont do it that way.

I do it a little simpler: viewtopic.php?f=21&t=28133 (using NSSM)
Tried the nssm method.
path: C:\Program Files\JAM Software\SpamAssassin for Windows\spamd.exe
Startup Directory: C:\Program Files\JAM Software\SpamAssassin for Windows
Arguments: -l -s "C:\Program Files (x86)\hMailServer\Logs\spamd.log" --max-spare=2 --max- children=20 --timeout-child=20
Name: SpamAssassin

When Start service, receive error as attached file, please help.
Attachments
Error.jpg
Error.jpg (25.42 KiB) Viewed 8966 times

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-26 13:02

UPDATE:

I HAVE GOT THE ANSWER.
l -s "C:\Program Files (x86)\hMailServer\Logs\spamd.log" --max-spare=2 --max- children=20 --timeout-child=20

There is a space in between which kills it. now the test is done. Thanks so much JIMI.

User avatar
jimimaseye
Moderator
Moderator
Posts: 8172
Joined: 2011-09-08 17:48

Re: SpamAssassin Questions

Post by jimimaseye » 2018-02-26 13:19

thomas10 wrote:
2018-02-26 13:02
UPDATE:

I HAVE GOT THE ANSWER.
l -s "C:\Program Files (x86)\hMailServer\Logs\spamd.log" --max-spare=2 --max- children=20 --timeout-child=20

There is a space in between which kills it. now the test is done. Thanks so much JIMI.
Yes, --max-children is a parameter and should be one full word.

Where did you get these instructions? Do they also show the misplaced space?
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-27 03:55

jimimaseye wrote:
2018-02-26 13:19
Yes, --max-children is a parameter and should be one full word.

Where did you get these instructions? Do they also show the misplaced space?
This is the link. By percepts. There is a space there.
viewtopic.php?p=164408#p165590

User avatar
mattg
Moderator
Moderator
Posts: 20294
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: SpamAssassin Questions

Post by mattg » 2018-02-27 04:33

fixed
Thanks
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

thomas10
Normal user
Normal user
Posts: 57
Joined: 2013-10-30 03:13

Re: SpamAssassin Questions

Post by thomas10 » 2018-02-27 09:52

mattg wrote:
2018-02-27 04:33
fixed
Thanks
You're welcome and thanks for fixing, matt.
I have successfully turn on SA and the Chinese text spam is finally out of sight. Thanks for the guidance.
I have set 'bayes_auto_learn 1' in local.cf but I am not sure whether the auto learn is running or not as the token is still the same without increasing space.
Below is the header of message marked as spam. Saw there is one autolearn=0, but not sure is it the one.
Return-Path: daiqfcoscol@gmail.com
X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on GCF
X-Spam-Flag: YES
X-Spam-Level: *******************
X-Spam-Status: Yes, score=19.5 required=5.0 tests=AXB_XMAILER_MIMEOLE_OL_024C2,
DKIM_ADSP_CUSTOM_MED,FORGED_MUA_OUTLOOK,FORGED_OUTLOOK_HTML,
FORGED_OUTLOOK_TAGS,FREEMAIL_FROM,FROM_MISSP_FREEMAIL,FROM_MISSP_MSFT,
FROM_MISSP_USER,FSL_NEW_HELO_USER,HK_NAME_FM_MR_MRS,HTML_MESSAGE,
MIME_HTML_ONLY,MISSING_HEADERS,MSGID_FROM_MTA_HEADER,NML_ADSP_CUSTOM_MED,
NSL_RCVD_FROM_USER,RCVD_IN_SORBS_WEB,RDNS_NONE,SPF_HELO_FAIL,SUBJ_ALL_CAPS,
TO_NO_BRKTS_FROM_MSSP,TO_NO_BRKTS_MSFT,ZMIde_OutlookExpress autolearn=no
autolearn_force=no version=3.4.1
X-Spam-Report: * 0.0 NSL_RCVD_FROM_USER Received from User * 1.6 SUBJ_ALL_CAPS Subject is
all capitals * 0.0 SPF_HELO_FAIL SPF: HELO does not match SPF record (fail)
* [SPF failed: Please see http://www.openspf.org/Why?s=helo;id=v1 ... 3.18;r=GCF]
* 1.2 MISSING_HEADERS Missing To: header * 0.0 FREEMAIL_FROM Sender
email is commonly abused enduser mail provider * (daiqfcoscol[at]gmail.com)
* 0.0 DKIM_ADSP_CUSTOM_MED No valid author signature, adsp_override is *
CUSTOM_MED * 1.1 MIME_HTML_ONLY BODY: Message only has text/html MIME
parts * 0.0 HTML_MESSAGE BODY: HTML included in message * 1.5
RCVD_IN_SORBS_WEB RBL: SORBS: sender is an abusable web server * [172.93.148.199
listed in dnsbl.sorbs.net] * 0.0 MSGID_FROM_MTA_HEADER Message-Id was
added by a relay * 0.0 AXB_XMAILER_MIMEOLE_OL_024C2 Yet another X header
trait * 0.6 FORGED_OUTLOOK_TAGS Outlook can't send HTML in this format *
1.0 ZMIde_OutlookExpress Outlook Express should not be used anymore * 0.0
FROM_MISSP_USER From misspaced, from "User" * 0.0 FROM_MISSP_MSFT From
misspaced + supposed Microsoft tool * 0.0 FORGED_OUTLOOK_HTML Outlook
can't send HTML message only * 1.3 RDNS_NONE Delivered to internal
network by a host with no rDNS * 1.5 HK_NAME_FM_MR_MRS No description
available. * 0.0 FSL_NEW_HELO_USER Spam's using Helo and User * 1.2
NML_ADSP_CUSTOM_MED ADSP custom_med hit, and not from a mailing list * 1.0
TO_NO_BRKTS_FROM_MSSP Multiple header formatting problems * 2.8
FORGED_MUA_OUTLOOK Forged mail pretending to be from MS Outlook * 2.5
TO_NO_BRKTS_MSFT To: lacks brackets and supposed Microsoft tool * 2.3
FROM_MISSP_FREEMAIL From misspaced + freemail provider
Received: from v17.te.com.com (Unknown [198.211.13.18]) by global-gp.com with ESMTP ; Tue,
27 Feb 2018 12:34:07 +0800
Message-ID: <3C1EFEEE-BBD9-47F8-A6D8-3EDA32A80E7C@global-gp.com>
Received: from User (unknown [172.93.148.199]) by v17.te.com.com (Postfix) with ESMTPA
id B192C5933; Mon, 26 Feb 2018 23:25:52 -0500 (EST)
From: ""Mr.Dai qianfeng (COSCO Shipping)" <daiqf@coscol.com.cn>"<daiqfcoscol@gmail.com>
Subject: [SPAM] [19.5] AGENT APPOINTMENT FOR MV LE YE V107 CALLING FOR DISCHARGING
Date: Mon, 26 Feb 2018 20:27:13 -0800
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="----=_NextPart_000_007F_01C2A9A6.15CA5A88"
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 6.00.2600.0000
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000
X-Spam-Prev-Subject: AGENT APPOINTMENT FOR MV LE YE V107 CALLING FOR DISCHARGING
X-hMailServer-Spam: YES
X-hMailServer-Reason-1: The host name specified in HELO does not match IP address. - (Score: 2)
X-hMailServer-Reason-2: Tagged as Spam by SpamAssassin - (Score: 5)
X-hMailServer-Reason-Score: 7
X-hMailServer-LoopCount: 1

User avatar
jimimaseye
Moderator
Moderator
Posts: 8172
Joined: 2011-09-08 17:48

Re: SpamAssassin Questions

Post by jimimaseye » 2018-02-27 10:26

Its not so straight forward yes or no.

https://wiki.apache.org/spamassassin/Au ... NotWorking

Tip: the spamassassin mail list is the best place for specific functionality questions. (I am subscribed for curiosity and it is always being answered by helpers).
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

Post Reply