SpamAssassin Score + hMailServer Score

Use this forum for discussions about SpamAssassin and anti-spam in general.
Post Reply
superman20
New user
New user
Posts: 29
Joined: 2015-03-05 03:10

SpamAssassin Score + hMailServer Score

Post by superman20 » 2015-03-05 03:18

Is it intended behavior that hMailServer will not add the SpamAssassin score to its own score unless the SpamAssassin score is greater than or equal to the SpamAssassin spam threshold (i.e., SpamAssassin tags it as spam)? I've seen some discussions on this and it is eventually just dropped and the people usually just lower their SpamAssassin threshold score to make hMailServer always add the scores together (thus making SpamAssassin tag virtually everything as spam). It seems more logical to always add the scores together (or at least give us the option). Is this by design or is it a bug? Are there people who actually want it to work that way it currently is?

Thanks,
Chad

User avatar
mattg
Moderator
Moderator
Posts: 20123
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: SpamAssassin Score + hMailServer Score

Post by mattg » 2015-03-05 06:34

If SpamAssassin tags mail as spam, does that change the message?
(I don't use SpamAssassin)

I'm also guessing that the spam score may NOT be relayed to hMailserver by SpamAssassin unless the message is marked as SPAM
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

superman20
New user
New user
Posts: 29
Joined: 2015-03-05 03:10

Re: SpamAssassin Score + hMailServer Score

Post by superman20 » 2015-03-05 06:52

When SpamAssassin scores it above the configured threshold, it will add an X-Spam-Status: YES header (along with other informative headers). I downloaded the hMailServer source code and verified that it does not store the SpamAssassin score (and thus pass it back up to main spam handling routine) unless it finds the X-Spam-Status: YES header. Unless most people really like this behavior, I propose that it always count the score regardless of the X-Spam-Status value. In fact, I feel like that is the whole point of scoring....you keep testing and keep adding up scores until your ultimate threshold is reached. In my particular case, SpamAssassin gives a score of 4.9 (where 5.0 is the SpamAssassin threshold) and then hMailServer failed the SPF test which I score as 5. The total score should have been 9.9, but hMailServer just scored it as 5. My delete threshold is 9, so the mail should have been deleted but it wasn't.

User avatar
mattg
Moderator
Moderator
Posts: 20123
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: SpamAssassin Score + hMailServer Score

Post by mattg » 2015-03-05 07:25

So if you had set your SpamAssassin score at 1, the actual score value would have been passed, and the message rejected.

From what you are saying, the down side to setting the SpamAssassin mark score to 1 is that some mail that doesn't reach the hmailsevrer spam score will contain a spamAssassin header showing a SPamAssassin score?
Is that such a big deal?
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
jimimaseye
Moderator
Moderator
Posts: 8125
Joined: 2011-09-08 17:48

Re: SpamAssassin Score + hMailServer Score

Post by jimimaseye » 2015-03-05 11:01

In my opinion I cant see why you would want to add SA score unless SA has deemed it potential spam.

By using SA you are trusting it and iuts rules to make a judgement: as per your SA configuration, a mail is either deemed as Spam or it is deemed as safe.

You kow that SA rules add scores in decimal and as negatives such as

0.7
-0.2
0.5
1.3
-1.3

and as such it collectively determines its spam value. Furthermore it does this because the rules and its abilities are FAR MORE advanced than any HMS does internally.

By using the SA score if only appropriate when you think SA may have determined as spam (I have the threshold set at 3 which seems to be just right) and yet you want to not FULLY trust it and add HMS tests on to it too. ie, maybe SA scores 3.1 (determining as SPAM), and you have your own HMS threshold set to something higher (5 or 6) allowing for your own HMS tests. Of course this your own HMS test scoring is probably not as fine tuned as the SA scoring rules so is more brute force.

The alternative you are proposing is to say that even though SA thinks a mail is not spam (because its only scored 1.0), you are going to use this '1' and add it to you own HMS test; well what happens if the SA 'HELO' yest is the score of 1, and then you run HMS 'HELO' test scoring 4 (same test yet scored twice?) You now deem your mail as spam (acheiving 5) when in reality both SA and HMS hasnt REALLY found it as spam at all. Whereas using the existing method the mail, even though itis being tested twice for the same condition, still isnt deemed as spam (SA scores it 1.0, hms scores it 4 - and yet your HMS score threshold is 5).

My point is that it is right in my mind to perform the way it does (only when SA considers it as spam) when you have already decided to trust SA's decision making and judgement.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
mattg
Moderator
Moderator
Posts: 20123
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: SpamAssassin Score + hMailServer Score

Post by mattg » 2015-03-05 14:03

jimimaseye wrote:Furthermore it does this because the rules and its abilities are FAR MORE advanced than any HMS does internally.
Agreed.
Thinking about this, I'd expect that the SpamAssassin score was added irrespective of whether SpamAssassin marked the message as SPAM or not. (How else could the negative values be useful) That's certainly how the GUI looks.
I'd think that NOT doing that is a bug, and tha this should be added to the issue tracker at https://github.com/hmailserver/hmailserver/issues
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
jimimaseye
Moderator
Moderator
Posts: 8125
Joined: 2011-09-08 17:48

Re: SpamAssassin Score + hMailServer Score

Post by jimimaseye » 2015-03-05 15:55

To give you an idea Matt, a typical Spamassassin header is added regardless and looks like this

Code: Select all

X-Spam-Status: No, score=0.3 required=3.0 tests=BAYES_00,
	DYN_RDNS_AND_INLINE_IMAGE,HTML_MESSAGE,RDNS_DYNAMIC,SPF_PASS,
	T_KAM_HTML_FONT_INVALID autolearn=no autolearn_force=no version=3.4.0
X-Spam-Report: 
	* -0.0 SPF_PASS SPF: sender matches SPF record
	*  0.0 T_KAM_HTML_FONT_INVALID BODY: Test for Invalidly Named or Formatted
	*      Colors in HTML
	* -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%
	*      [score: 0.0000]
	*  0.0 HTML_MESSAGE BODY: HTML included in message
	*  1.0 RDNS_DYNAMIC Delivered to internal network by host with
	*      dynamic-looking rDNS
	*  1.2 DYN_RDNS_AND_INLINE_IMAGE Contains image, and was sent by dynamic
	*      rDNS
	*
where everything from "tests=" are the names of all rules applied and scored due to matching. The spam 'report' then lists them test individually with their scores.

Now, this is a good example: given this particular report scored overall 0.3, how would you have HMS take that score (as it only deals with integer scores)?


And here is another example:

Code: Select all

X-Spam-Status: No, score=-4.4 required=3.0 tests=BAYES_00,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,HTML_MESSAGE,KHOP_RCVD_TRUST,
	RCVD_IN_DNSWL_LOW,RCVD_IN_HOSTKARMA_YE,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,
	SPF_PASS,T_KAM_HTML_FONT_INVALID autolearn=ham autolearn_force=no
	version=3.4.0
X-Spam-Report: 
	*  0.0 RCVD_IN_HOSTKARMA_YE RBL: HostKarma: relay in yellow list (varies)
	*      [209.85.212.177 listed in hostkarma.junkemailfilter.com]
	*  0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
	*      (sandimy[at]gmail.com)
	* -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low
	*      trust
	*      [209.85.212.177 listed in list.dnswl.org]
	* -0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3)
	*      [209.85.212.177 listed in wl.mailspike.net]
	* -0.0 SPF_PASS SPF: sender matches SPF record
	*  0.0 T_KAM_HTML_FONT_INVALID BODY: Test for Invalidly Named or Formatted
	*      Colors in HTML
	* -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%
	*      [score: 0.0000]
	*  0.0 HTML_MESSAGE BODY: HTML included in message
	* -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's
	*       domain
	* -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
	*  0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily
	*      valid
	* -0.0 RCVD_IN_MSPIKE_WL Mailspike good senders
	* -1.8 KHOP_RCVD_TRUST DNS-Whitelisted sender is verified
	*
The result of this is is MINUS 4.4 (-4.4). Now if you were to apply your own HMS rules in DNSBL or SURBL (that SA doesnt cover) and even score a match at 5 and 4 (total 9) it would still not hit a HMS threshold of 5 (which you may have set) - despite HMS actually scoring way and above this.

This explains why I belive you should only use SA scores when SA has determined it as spam by hitting ITS threshold.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
SorenR
Senior user
Senior user
Posts: 3184
Joined: 2006-08-21 15:38
Location: Denmark

Re: SpamAssassin Score + hMailServer Score

Post by SorenR » 2015-03-05 16:35

jimimaseye wrote:... or SURBL (that SA doesnt cover) ...
Mine does... Hint: "URIBL" :wink:

Code: Select all

X-Spam-Status: Yes, score=44.5 required=3.0 tests=BAYES_99,BAYES_999, 
   BODY_URI_ONLY,KAM_RBL,KAM_VERY_BLACK_DBL,MSGID_FROM_MTA_HEADER, 
   RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK,
   RCVD_IN_BL_SPAMCOP_NET,RCVD_IN_BRBL_LASTEXT,RCVD_IN_MSPIKE_BL,
   RCVD_IN_MSPIKE_L5,RCVD_IN_PBL,RCVD_IN_PSBL,RCVD_IN_RP_RNBL,RCVD_IN_SORBS_WEB,
   RCVD_IN_XBL,RCVD_NUMERIC_HELO,TVD_RCVD_IP,TVD_RCVD_IP4,T_FSL_HELO_BARE_IP_2,
   URIBL_AB_SURBL,URIBL_BLACK,URIBL_DBL_SPAM,URIBL_JP_SURBL,URIBL_SBL,
   URIBL_SBL_A,URIBL_SC_SURBL,URIBL_WS_SURBL autolearn=disabled version=3.4.0
X-Spam-Report: 
   *  0.6 URIBL_SC_SURBL Contains an URL listed in the SC SURBL blocklist 
   *      [URIs: hotdrugsstore.in] 
   *  1.3 URIBL_JP_SURBL Contains an URL listed in the JP SURBL blocklist 
   *      [URIs: hotdrugsstore.in] 
   *  4.5 URIBL_AB_SURBL Contains an URL listed in the AB SURBL blocklist 
   *      [URIs: hotdrugsstore.in]
   *  1.6 URIBL_WS_SURBL Contains an URL listed in the WS SURBL blocklist 
   *      [URIs: hotdrugsstore.in] 
   *  3.3 RCVD_IN_PBL RBL: Received via a relay in Spamhaus PBL 
   *      [109.135.11.38 listed in zen.spamhaus.org] 
   *  0.4 RCVD_IN_XBL RBL: Received via a relay in Spamhaus XBL 
   *  3.5 BAYES_99 BODY: Bayes spam probability is 99 to 100% 
   *      [score: 1.0000] 
   *  0.0 TVD_RCVD_IP Message was received from an IP address 
   *  0.0 TVD_RCVD_IP4 Message was received from an IPv4 address 
   *  1.2 RCVD_NUMERIC_HELO Received: contains an IP address used for HELO 
   *  0.2 BAYES_999 BODY: Bayes spam probability is 99.9 to 100% 
   *      [score: 1.0000] 
   *  1.9 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level 
   *  above 50% 
   *      [cf: 100] *  0.9 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)
   *  0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50% 
   *      [cf: 100] 
   *  2.5 URIBL_DBL_SPAM Contains a spam URL listed in the DBL blocklist 
   *      [URIs: hotdrugsstore.in] 
   *  1.7 URIBL_BLACK Contains an URL listed in the URIBL blacklist 
   *      [URIs: hotdrugsstore.in] 
   *  3.2 RCVD_IN_MSPIKE_L5 RBL: Very bad reputation (-5) 
   *      [109.135.11.38 listed in bl.mailspike.net] 
   *  1.4 RCVD_IN_BRBL_LASTEXT RBL: No description available. 
   *      [109.135.11.38 listed in bb.barracudacentral.org] 
   *  2.7 RCVD_IN_PSBL RBL: Received via a relay in PSBL 
   *      [109.135.11.38 listed in psbl.surriel.com] 
   *  0.8 RCVD_IN_SORBS_WEB RBL: SORBS: sender is an abusable web server 
   *      [109.135.11.38 listed in dnsbl.sorbs.net] 
   *  1.3 RCVD_IN_RP_RNBL RBL: Relay in RNBL, 
   *      https://senderscore.org/blacklistlookup/
   *      [109.135.11.38 listed in bl.score.senderscore.com] 
   *  1.3 RCVD_IN_BL_SPAMCOP_NET RBL: Received via a relay in bl.spamcop.net 
   *      [Blocked - see <http://www.spamcop.net/bl.shtml?109.135.11.38>] 
   *  0.1 URIBL_SBL_A Contains URL's A record listed in the SBL blocklist 
   *      [URIs: meekly.hotdrugsstore.in]
   *  1.6 URIBL_SBL Contains an URL's NS IP listed in the SBL blocklist 
   *      [URIs: meekly.hotdrugsstore.in] 
   *  2.0 KAM_RBL Higher scores for hitting multiple trusted RBLs 
   *  0.0 RCVD_IN_MSPIKE_BL Mailspike blacklisted 
   *  5.0 KAM_VERY_BLACK_DBL Email that hits both URIBL Black and Spamhaus DBL 
   *  0.0 MSGID_FROM_MTA_HEADER Message-Id was added by a relay 
   *  0.0 T_FSL_HELO_BARE_IP_2 No description available. 
   *  1.0 BODY_URI_ONLY Message body is only a URI in one line of text or for 
   *      an image
I did a quick search on my server for the highest SPAM score in the last 6 months... 66.2 and it passed ALL of the other SPAM tests in hMailServer, except SpamAssassin ... :shock:
SørenR.

“With age comes wisdom, but sometimes age comes alone.”
- Oscar Wilde

User avatar
jimimaseye
Moderator
Moderator
Posts: 8125
Joined: 2011-09-08 17:48

Re: SpamAssassin Score + hMailServer Score

Post by jimimaseye » 2015-03-05 16:46

SorenR wrote:Mine does... Hint: "URIBL" :wink:
I meant you could be adding your OWN lookups into HMS (and setting your own scores against positive matches for them) that Spamassassin has not been coded/rule defined to cover. (I know SA covers most of the main/popular ones such as multi.surbl.org etc., but there might be one the user has found that is not covered by SA rules that he chooses to add).
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
SorenR
Senior user
Senior user
Posts: 3184
Joined: 2006-08-21 15:38
Location: Denmark

Re: SpamAssassin Score + hMailServer Score

Post by SorenR » 2015-03-05 16:48

jimimaseye wrote:
SorenR wrote:Mine does... Hint: "URIBL" :wink:
I meant you could be adding your OWN lookups into HMS (and setting your own scores against positive matches for them) that Spamassassin has not been coded/rule defined to cover. (I know SA covers most of the main/popular ones such as multi.surbl.org etc., but there might be one the user has found that is not covered by SA rules that he chooses to add).
Ah.. My bad :oops:
SørenR.

“With age comes wisdom, but sometimes age comes alone.”
- Oscar Wilde

superman20
New user
New user
Posts: 29
Joined: 2015-03-05 03:10

Re: SpamAssassin Score + hMailServer Score

Post by superman20 » 2015-03-05 17:18

SpamAssassin can be configured to use other URIBL and DNSBL that aren't provide out of the box. I do, in fact, do this. In my configuration, I was assuming that spam scores were all additive and so I disable the DNS and SURBL options in hMailServer so that they would not contribute to double scoring. I have also spent time slowly studying the types of spam we receive and carefully tuning SpamAssassin to my exact needs. I'm somewhat new to hMailServer...coming from using other commercial products (SecurityGateway, SpamTitan, ORF, etc). All of the other commercial products having a continuous running spam score and I have found this to be very logical and effective. I let hMailServer continue to do SPF, HELO command, and sender DNS-MX checks because it is better suited to do so. I feel like those checks combined with the SpamAssassin checks provide very accurate spam checks. In fact, the majority of spams that get through to my system are the ones where SpamAssassin scores below 5 and hMailServer ignores the score (but would have been spam if it didn't because it failed an hMailServer test).

I also feel like hMailServer should change to all floating point scoring like some of the other commercial solutions so that there wouldn't be any integer truncation when adding everything together.

jimimaseye, your example that scored negative is a bit biased. Part of the negative contribution was because the mail is in some trusted whitelist databases. In my experience, you don't find the same IP's and domains both in blacklists AND whitelists....so, while not impossible, it is statistically unlikely the any DNSBL or SURBL hits would have fired for your example.

User avatar
jimimaseye
Moderator
Moderator
Posts: 8125
Joined: 2011-09-08 17:48

Re: SpamAssassin Score + hMailServer Score

Post by jimimaseye » 2015-03-05 19:04

For me, the functionality seems logical to me (as I tried to explain above). The spamassassin score can be 'used', or simply the spam=yes/no and your own score applied.

I suspect it was intended to use EITHER spamassassin scores ONLY thereby leaving all decision making up to SA and not adding to by your own rules, OR you use SA conclusion ("spam=yes") and score it yourself along with having your own HMS testing (HELO, SPF, DNSBL etc). It seems that using SA score and then adding HMS test score to it isnt what it was intended for. After all, why would you ask SA to test for something, then do exactly the same test again in HMS (effectively doubling up the scoring probability) which is a scenario that is VERY possible (we have already identified the Spamassassin does the SPF, HELO, mainstream DNSBL and SURBL tests anyway - so why ask HMS to double check and double the points if you are choosing to accept SA scoring?)

Ideally, you would simply 'USE SPAMASSASSIN SCORING=YES', set your threshold as such (as you do in SA) and leave all HMS scoring and testing turned off (with exception of you having a specific surbl/dnsbl test that you KNOW your SA rules are not covering).
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
SorenR
Senior user
Senior user
Posts: 3184
Joined: 2006-08-21 15:38
Location: Denmark

Re: SpamAssassin Score + hMailServer Score

Post by SorenR » 2015-03-05 22:38

Spam scoring is a regional thing, I don't see the same SPAM as everyone else thus my rules should ideally be different and scored differently.

Unless you are a SpamAssassin expert (or sufficiently nerdy) you can create your own rules to grade scoring to match your environment - most choose not to and rely on pre-built rules that are updated based on a world average of SPAM, simply based on lack of time to maintain those rules. SPAM is evolving all the time and so should the rules to catch it.

Most spammers break rules to deliver SPAM in the most efficient way possible - because someone is paying them - and as we all know; Time is Money.

- GreyListing is a powerfull tool - This is where "Time is Money" become important.
- SPF WAS a powerfull tool, it's use is increasing amongst spammers.
- DKIM make everything just a bit more reliable.
- HELO is unfortunately not a reliable way to identify spammers as more home users are "in-sourcing" their mail systems.
- MX... Well, this is where WE can break the rules. It is not a requirement to have MX records according to the RFC's. But, we believe any respectable IT department would have them, if for nothing else than to fight SPAM with a blackhole.MX setup.
- RBL's and SURBL's is a matter of choice. Find one (or more) you trust and "get an opinion from a trusted source".

SpamAssassin will do nearly all of the above, maybe not with the scoring we'd like to use, but then we can add them to hMailServer ourselves. It's like fine tuning SpamAssassin, outside of SpamAssassin.

Add it all together, and we'll get a pretty good picture of what is SPAM, and what is HAM.

For my part I rate everything as 3, SpamAssassin triggers at 3.0. Anything 3 or above is marked as SPAM, moved into the users SPAM folder, and forwarded to a dedicated SPAM user for further analysis - if needed. Only SPAM scored above 100 is deleted/rejected.
False-positives are added to a hMail rule-based whitelist to prevent them being treated as SPAM - however they will still be marked as SPAM.

Each users SPAM folder and INBOX folder is processed every night to maintain the Bayesian database used by SpamAssassin. The intended rationale is to "localize" SpamAssassin to my neck of the woods. Also the users are able to partly influence classification by moving emails between the two folders for processing the next night (actually, they have a 30 day window).

Despite all efforts I do get SPAM that only SpamAssassin catch... Spammers are getting increasingly cleverer.

As I posted earlier...
I did a quick search on my server for the highest SPAM score in the last 6 months... 66.2 and it passed ALL of the other SPAM tests in hMailServer (Greylist, SPF, DKIM, HELO, MX, 4 RBL's and SURBL), except SpamAssassin ... :shock:
SørenR.

“With age comes wisdom, but sometimes age comes alone.”
- Oscar Wilde

User avatar
jimimaseye
Moderator
Moderator
Posts: 8125
Joined: 2011-09-08 17:48

Re: SpamAssassin Score + hMailServer Score

Post by jimimaseye » 2015-03-05 23:40

Just for info, my SA has a mark threshold of 3 and that's when mail gets marked as [SPAM].

Anything that reaches HMS over a score of 7 is automatically deleted - remaining unseen and unknown (technically not true - it gets moved to Trash folder straight away by a rule, so viewable IF you want to go there and see it).

I would say that 98% (if not more) of my mail comes in clean and unmarked or deleted as definite spam correctly. The other 2% gets marked as spam (by SA) but remains genuine (usually because the mail comes in sent with full capitals subjects and/or body content - spamassassin dont like that.)
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

superman20
New user
New user
Posts: 29
Joined: 2015-03-05 03:10

Re: SpamAssassin Score + hMailServer Score

Post by superman20 » 2015-03-06 04:19

Spam fighting is an art and a never ending battle. I feel like all available spam technologies should be employed. I like the design of being able to deploy anti-spam technologies in a score fashion and add everything together to make the final decision.

Not too many people have chimed in one way or the other. If I am the only one who really feels strongly that all methods should add together to one final score, then I'll just modify the source to behave like I want. After inspecting the source, it seems this change would be rather easy to make. I was really hoping the developer and community would feel as strongly as I do...as I hate maintaining a forked project.

It looks as though I could also write a script to grab the SpamAssassin score and the HMS score and add them together myself in the situations where HMS doesn't add them itself.

User avatar
mattg
Moderator
Moderator
Posts: 20123
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: SpamAssassin Score + hMailServer Score

Post by mattg » 2015-03-06 04:50

mattg wrote:I'd think that NOT doing that is a bug, and that this should be added to the issue tracker at https://github.com/hmailserver/hmailserver/issues
Martin doesn't spend a lot of time any more on the forum
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

superman20
New user
New user
Posts: 29
Joined: 2015-03-05 03:10

Re: SpamAssassin Score + hMailServer Score

Post by superman20 » 2015-03-06 05:00

Thanks. I'll try that and see where it goes.

User avatar
SorenR
Senior user
Senior user
Posts: 3184
Joined: 2006-08-21 15:38
Location: Denmark

Re: SpamAssassin Score + hMailServer Score

Post by SorenR » 2015-03-06 16:41

superman20 wrote:Spam fighting is an art and a never ending battle. I feel like all available spam technologies should be employed. I like the design of being able to deploy anti-spam technologies in a score fashion and add everything together to make the final decision.

Not too many people have chimed in one way or the other. If I am the only one who really feels strongly that all methods should add together to one final score, then I'll just modify the source to behave like I want. After inspecting the source, it seems this change would be rather easy to make. I was really hoping the developer and community would feel as strongly as I do...as I hate maintaining a forked project.

It looks as though I could also write a script to grab the SpamAssassin score and the HMS score and add them together myself in the situations where HMS doesn't add them itself.
If you do change the source code, you could try submitting it to Martin for review. You may get lucky and have it included in the release. :mrgreen:

Scripting is quite easy. I have my Backup-MX hosted with my ISP and they use a round-robin approach to DNS so the HELO check fails on 2 of 3 rDNS lookups :roll: also the DKIM check fails for obvious reasons, so I rewrite/recalculate the "X-hMailServer-Spam" and "X-hMailServer-Reason-Score" headers in those cases.
SørenR.

“With age comes wisdom, but sometimes age comes alone.”
- Oscar Wilde

User avatar
jimimaseye
Moderator
Moderator
Posts: 8125
Joined: 2011-09-08 17:48

Re: SpamAssassin Score + hMailServer Score

Post by jimimaseye » 2015-03-06 17:15

(Sorry I missed this and didnt answer earlier.)
superman20 wrote:jimimaseye, your example that scored negative is a bit biased. Part of the negative contribution was because the mail is in some trusted whitelist databases. In my experience, you don't find the same IP's and domains both in blacklists AND whitelists....so, while not impossible, it is statistically unlikely the any DNSBL or SURBL hits would have fired for your example.
In my example, I showed a scenario where a message ended up as MINUS score. Now, lets say that it was sent from china (it wssnt but it could have been). Still genuine, still allowed, not technically spam (hence its score). BUT....I have a DNSBL rule (zz.countries.nerd.dk) that scores anything coming from china a value of 8 which would be enough to reject this email due to hitting my 'delete' threshold of 8 (because I dont want anything from china). Any yet, in this example CLEARLY it would have been allowed in because -4.4+8 is only 3.6 = FAILED.

There is nothing unlikely about this scenario for people that are using such geo-blocking (as I am).
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

superman20
New user
New user
Posts: 29
Joined: 2015-03-05 03:10

Re: SpamAssassin Score + hMailServer Score

Post by superman20 » 2015-03-12 02:52

jimimaseye, I certainly appreciate the point you're trying to make, but your new example is a bit contradictory. You have negative points because your e-mail hits some whitelists and some positive points because the e-mail hits some blacklists. I don't think any spam configuration would properly deal with that sort of conflicting information. I actually implement your example somewhat but deal with it differently. My settings have e-mail that is geo-located from China to automatically score the reject/delete score...BUT I also make sure that these custom "extreme" rules are run first and when they hit then everything else is short-circuited. This prevents me from allowing a legitimate e-mail from getting any negative points when I want all China e-mail blocked (good or bad).

User avatar
jimimaseye
Moderator
Moderator
Posts: 8125
Joined: 2011-09-08 17:48

Re: SpamAssassin Score + hMailServer Score

Post by jimimaseye » 2015-03-12 11:40

superman20 wrote:BUT I also make sure that these custom "extreme" rules are run first and when they hit then everything else is short-circuited.
As is my case. And I dont need any special 'coding'/methods to ensure everything else is shortcircuited as this is just how things work currently. As my geoblock DNSBL is in the HMS and would hit the threshold it then gets rejected immediately and not passed to SA (delivery refused). However, your earlier suggestion is that everything should be added together so logically you wouldnt be able to shortcut because after the HMS performs its internal checks it would HAVE to call SA and get its scoring before it can concluded and react on its final scoring.

You cant have it both ways.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

superman20
New user
New user
Posts: 29
Joined: 2015-03-05 03:10

Re: SpamAssassin Score + hMailServer Score

Post by superman20 » 2015-03-12 21:06

You can somewhat have it both ways if you have spam systems that works together and not independently. Spam checking can definitely stop as soon as the delete threshold is reached. So if the HMS implemented checks hit the delete threshold, then there is no need to call any other checks. However, you must keep going down the chain calling all checks until the delete threshold is reached. Spam testing will never be absolute which is why I strongly feel that it must always be additive. You are adding probabilities and confidence levels that something is spam. If your spam level is 5 and HMS scores 4 and SpamAssassin scores 4 (and assuming a sensible setup where there are no redundant tests), then each one independently says NOT spam, but I'd be willing to bet that it is spam in almost all of those situations.

User avatar
jimimaseye
Moderator
Moderator
Posts: 8125
Joined: 2011-09-08 17:48

Re: SpamAssassin Score + hMailServer Score

Post by jimimaseye » 2015-03-12 22:16

the problem is that you are suggesting that two totally separate system, each with their own intensities and complexities (with SA being WAY more advanced than HMS), are somewhat 'collated' and the scores shared despite one being the little runt of the spam checking fraternity wholst the other one being the guru. Even if HMS scoring would allow negative scoring, it would make a LITTLE (just) more advanced and more like SA capabilities but it doesnt (SA realises there are positives, and then reasons to double check to apply negatives to counteract - something that HMS spam checking doesnt).

I maintain the two are TOTALLY separate systems (one being written and designed by HMS author and the other being created and written by unrelated entities who HMS author has no control over), it was designed to be that way (use on or the other but not both (although it wont stop you)) and was designed that way for a reason (recognition that SA will do the job a LOT better than HMS can ever dream of). Using ACTUAL SA scores and adding them to HMS scoring of internal checks wouldnt make sense because SA's idea of what scoring values work and what they should be are coded, tested, retested, fine-tuned, modified and implemented after a retest again. HMS scoring is simply (usually) choose a number, an INTEGER only at that, and apply it. Example, SA SPF fail check might score only 0.2 whereas in HMS its default is 2 or 3. Well how can that work together? Still, it will let you but only once it has taken the advice from the guru of spam-checking (Spamassassin) to decide on whether there is any REAL threat.

Thats my view anyway. :-)
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
mattg
Moderator
Moderator
Posts: 20123
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: SpamAssassin Score + hMailServer Score

Post by mattg » 2015-12-30 06:01

Just using some quiet time to implement SpamAssassin

New Windows 10 Pro machine. Enabled HyperV Server and created a new Ubuntu Server install (running on one core and 512 MB RAM) to run SpamAssassin, ClamAV as per >> viewtopic.php?f=21&t=29053

I've found SA marks far lower than I would like.
Playing with SA rules is deep nerdy stuff. I don't want to go and re-score all tests, and potentially break updates, so I have created a new SA rule that simply adds 2.2 to all SA scores. I have set SA to mark as spam if 3 or higher, without changing the subject.

My existing hMailserver AntiSPAM was working pretty good, with a mark at 5 and delete at 60

I've added ClamAV scores including the Sane-Security databases to SA. Currently Mail gets scanned twice by ClamAV, once to score, and then to categorically detect virus - I am watching to see how that works out.

Also using lots of additional databases and filters that I have found here.

Still looking for a good GeoIP addition for Spamassassin.
Still fine-tuning, but catching more SPAM without catching more HAM
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
RvdH
Senior user
Senior user
Posts: 798
Joined: 2008-06-27 14:42
Location: Netherlands

Re: SpamAssassin Score + hMailServer Score

Post by RvdH » 2015-12-30 13:48

@mattg

A nice source of DNB's lists ready to use with SpamAssassin is listed here:

http://www.intra2net.com/en/support/antispam/index.php
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

kroberts
New user
New user
Posts: 5
Joined: 2016-07-28 16:11

Re: SpamAssassin Score + hMailServer Score

Post by kroberts » 2017-06-13 15:35

Could someone post a copy of your spamassassin local.cf with your preferred rules to allow spamassassin do all of the dnsbl and uribl tests . I would like to move all the spam test to spamassassin for better implementation of the scoring and remove it out of hmailserver. It is very confusing when the 2 scoring systems either do not add together or counter each other. I say let one system score for spam and maybe hmailserver do the early spf and dns tests unless spamassassin can do those as well. Not being able to fine tune hmailserver except in whole number integers also skews the scoring. 4.9 is truncated to 4. Thank you

User avatar
jimimaseye
Moderator
Moderator
Posts: 8125
Joined: 2011-09-08 17:48

Re: SpamAssassin Score + hMailServer Score

Post by jimimaseye » 2017-06-13 17:15

My setup is here: viewtopic.php?f=21&t=28133 (personal settings are in the 2nd post). You will see I simply set a 'tagged by SA' as 5 in line with the builtin antispam tests. (You dont have to use SA's scoring system).

You can simply exclusive use SA if you wish by just disabling any builtin antispam tests (DNSBL's etc). I personally (as you will see) use a combination of both. SA does a far better job of antispam testing so you can be confident that in the main if HMS would find it then SA has already found it...and then some.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

kroberts
New user
New user
Posts: 5
Joined: 2016-07-28 16:11

Re: SpamAssassin Score + hMailServer Score

Post by kroberts » 2017-06-13 19:21

Thank you for your post. I had seen that setup but I was hoping that there would be a way of just controlling the dnsbl and uribl tests in the local.cf or another file without having to get into all the scripting. Not being a programmer, scripting get confusing if you do not use it all the time, at least for me. Is there a way to set it in the local.cf? Thank you for the help.

User avatar
SorenR
Senior user
Senior user
Posts: 3184
Joined: 2006-08-21 15:38
Location: Denmark

Re: SpamAssassin Score + hMailServer Score

Post by SorenR » 2017-06-13 20:51

In c$\SpamAssassin\share\3.004000\updates_spamassassin_org you will find two files; 20_dnsbl_tests.cf and 25_uribl.cf

The clever thing with SpamAssassin is that it reads all config files alphabetically ... So if you copy these two files to c$\SpamAssassin\etc\spamassassin and name them my_dnsbl_tests.cf and my_uribl.cf you can modify them all you want or change the score as they are read AFTER the originals.

Anyways, take a look at the files and you'll get the idea how to build your own lists.

***** Example *****

I have a config (KAM.cf) about 288 kb... There is this one rule ...

Code: Select all

#Bad UTF-8 content type and transfer encoding - Thanks to Pedro David Marco for alerting to issue
header	 __KAM_BAD_UTF8_1		Content-Type =~ /text\/html; charset=\"utf-8\"/i
header   __KAM_BAD_UTF8_2		Content-Transfer-Encoding =~ /base64/i
full	 __RW_BAD_UTF8_3 		/^(?:[^\n]|\n(?!\n))*\nContent-Transfer-Encoding:\s+base64(?:[^\n]|\n(?!\n))*\n\n[\s\n]{0,300}[^\s\n].{0,300}[^a-z0-9+\/=\n][^\s\n]/si

meta	KAM_BAD_UTF8	(__KAM_BAD_UTF8_1 + __KAM_BAD_UTF8_2 + __RW_BAD_UTF8_3 >= 3)
score	KAM_BAD_UTF8	14.0
describe KAM_BAD_UTF8	Bad Content Type and Transfer Encoding that attempts to evade SA scanning
that check the entire message incl. attachments. If someone send an email to me with a PDF file in it, it usually takes 300+ seconds and then hMail fails.

I have created an extra config (KAM-fix.cf) containing only this rule

Code: Select all

#Bad UTF-8 content type and transfer encoding - Thanks to Pedro David Marco for alerting to issue
header	 __KAM_BAD_UTF8_1		Content-Type =~ /text\/html; charset=\"utf-8\"/i
header   __KAM_BAD_UTF8_2		Content-Transfer-Encoding =~ /base64/i
body	 __RW_BAD_UTF8_3 		/^(?:[^\n]|\n(?!\n))*\nContent-Transfer-Encoding:\s+base64(?:[^\n]|\n(?!\n))*\n\n[\s\n]{0,300}[^\s\n].{0,300}[^a-z0-9+\/=\n][^\s\n]/si

meta	KAM_BAD_UTF8	(__KAM_BAD_UTF8_1 + __KAM_BAD_UTF8_2 + __RW_BAD_UTF8_3 >= 3)
score	KAM_BAD_UTF8	14.0
describe KAM_BAD_UTF8	Bad Content Type and Transfer Encoding that attempts to evade SA scanning
where "full" is replaced with "body" in line 4. Since KAM.cf is read first and then KAM-fix.cf, it changes the rule. Now everything passes in less than 10 seconds. - And I don't have to create a script to alter the file every time it is auto-updated.
SørenR.

“With age comes wisdom, but sometimes age comes alone.”
- Oscar Wilde

kroberts
New user
New user
Posts: 5
Joined: 2016-07-28 16:11

Re: SpamAssassin Score + hMailServer Score

Post by kroberts » 2017-06-13 21:51

Great. I will look at implementing this. Thank you for sharing your knowledge.

kroberts
New user
New user
Posts: 5
Joined: 2016-07-28 16:11

Re: SpamAssassin Score + hMailServer Score

Post by kroberts » 2017-06-13 22:08

In your experience with whitelisting and blacklisting is there an easy manageable way to add a whitelist/blacklists in spamassassin instead of hmailserver? I like hmailserver fine but not easy to manage the whitelist and blocking rules when they grow like mine have since trying to get a handle on all the different ways to stop spam but not stop ham. I end up adding the same line or rule again and again. I am sure in your experiences you have said I think there is an easier way to to implement this or manage this. Thanks in advance.

User avatar
SorenR
Senior user
Senior user
Posts: 3184
Joined: 2006-08-21 15:38
Location: Denmark

Re: SpamAssassin Score + hMailServer Score

Post by SorenR » 2017-06-14 00:06

kroberts wrote:In your experience with whitelisting and blacklisting is there an easy manageable way to add a whitelist/blacklists in spamassassin instead of hmailserver? I like hmailserver fine but not easy to manage the whitelist and blocking rules when they grow like mine have since trying to get a handle on all the different ways to stop spam but not stop ham. I end up adding the same line or rule again and again. I am sure in your experiences you have said I think there is an easier way to to implement this or manage this. Thanks in advance.
My SpamAssassin is reasonable well trained after 3 years so I have only a few addresses whitelisted in SpamAssassin.
I don't have a blacklist per se... I block emails on multiple levels of identification; body, from, helo and subject, all done in eventhandlers; OnClientConnect(oClient), OnHELO(oClient) and OnAcceptMessage(oClient, oMessage).
80% of what I block is rejected, the rest is marked as SPAM and my daily SpamAssassin training eventually learn the blacklisted emails so I can clean some of the manual blacklist after about 1 month or so.

I check my custom logs every day and adjust filters if needed. Last time was IIRC 2 weeks ago - and I also built a new IDS function to catch brute force IMAPS logon attempts a few days ago.
SørenR.

“With age comes wisdom, but sometimes age comes alone.”
- Oscar Wilde

User avatar
mattg
Moderator
Moderator
Posts: 20123
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: SpamAssassin Score + hMailServer Score

Post by mattg » 2017-06-14 00:59

Here is a snippet of my custom.cf
(I changed the name so that it wasn't overwritten on SpamAssassin upgrade)

Code: Select all

#   Some shortcircuiting, if the plugin is enabled
# 
ifplugin Mail::SpamAssassin::Plugin::Shortcircuit
#
#   default: strongly-whitelisted mails are *really* whitelisted now, if the
#   shortcircuiting plugin is active, causing early exit to save CPU load.
#   Uncomment to turn this on
#
shortcircuit USER_IN_WHITELIST       on
# shortcircuit USER_IN_DEF_WHITELIST   on
shortcircuit USER_IN_ALL_SPAM_TO     on
shortcircuit SUBJECT_IN_WHITELIST    on

#   the opposite; blacklisted mails can also save CPU
#
shortcircuit USER_IN_BLACKLIST       on
# shortcircuit USER_IN_BLACKLIST_TO    on
# shortcircuit SUBJECT_IN_BLACKLIST    on

#   if you have taken the time to correctly specify your "trusted_networks",
#   this is another good way to save CPU
#

endif # Mail::SpamAssassin::Plugin::Shortcircuit


# don't score URIBL
score URIBL_BLACK 0
score URIBL_RED 0
score URIBL_GREY 0
score URIBL_BLOCKED 0

# DNSBL scores
score URIBL_DBL_SPAM 4
score RCVD_IN_SBL 3


# blacklist from
blacklist_from *.top
blacklist_from *.eu
blacklist_from *.download
blacklist_from *.accountant
blacklist_from *.cf
blacklist_from *.party
blacklist_from *.review
blacklist_from *.faith
blacklist_from *.win
blacklist_from *.trade
blacklist_from *.webcam
blacklist_from *.racing
blacklist_from *.date
blacklist_from *.bid
blacklist_from *.cricket

# whitelist from
whitelist_from *@important_domain.com.au


## BELOW is my ClamAV Integration
## NOT needed for what you are doing


loadplugin ClamAV clamav.pm
full CLAMAV eval:check_clamav()
describe CLAMAV Clam AntiVirus detected something...
score CLAMAV 0.001

# Look for specific types of ClamAV detections
header __CLAMAV_PHISH X-Spam-Virus =~ /Yes.{1,30}Phishing/i
header __CLAMAV_PHISH_HEUR X-Spam-Virus =~ /Yes.{1,30}Phishing\.Heuristics\.Email/
header __CLAMAV_SANE X-Spam-Virus =~ /Yes.{1,30}Sanesecurity/i
header __CLAMAV_MBL X-Spam-Virus =~ /Yes.{1,30}MBL/
header __CLAMAV_MSRBL X-Spam-Virus =~ /Yes.{1,30}MSRBL/
header __CLAMAV_VX X-Spam-Virus =~ /Yes.{1,30}VX\./

# Give the above rules a very late priority so that they can see the output
# of previous rules - otherwise they don't work! Not sure what the correct
# priority should be but this seems to work...
priority __CLAMAV_PHISH 9999
priority __CLAMAV_PHISH_HEUR 9999
priority __CLAMAV_SANE 9999
priority __CLAMAV_MBL 9999
priority __CLAMAV_MSRBL 9999
priority __CLAMAV_VX 9999


# Work out what ClamAV detected and score accordingly

# ClamAV general signatures
meta CLAMAV_VIRUS (CLAMAV && !__CLAMAV_PHISH && !__CLAMAV_SANE && !__CLAMAV_MBL && !__CLAMAV_MSRBL && !__CLAMAV_VX)
describe CLAMAV_VIRUS Virus found by ClamAV default signatures
score CLAMAV_VIRUS 20.0

# ClamAV phishing signatures
meta CLAMAV_PHISH (CLAMAV && __CLAMAV_PHISH && !__CLAMAV_SANE && !__CLAMAV_PHISH_HEUR)
describe CLAMAV_PHISH Phishing email found by ClamAV default signatures
score CLAMAV_PHISH 10.0

# ClamAV phishing with heuristic engine (not signatures based, may lead to false positives)
# Available since ClamAV 0.91
meta CLAMAV_PHISH_HEUR (CLAMAV && __CLAMAV_PHISH_HEUR)
describe CLAMAV_PHISH_HEUR Phishing email found by ClamAV heuristic engine
score CLAMAV_PHISH_HEUR 2.0

# ClamAV SaneSecurity signatures from http://www.sanesecurity.com/clamav/
meta CLAMAV_SANE (CLAMAV && __CLAMAV_SANE)
describe CLAMAV_SANE SPAM found by ClamAV SaneSecurity signatures
score CLAMAV_SANE 15

# ClamAV MBL signatures from http://www.malware.com.br/
meta CLAMAV_MBL (CLAMAV && __CLAMAV_MBL)
describe CLAMAV_MBL Malware found by ClamAV MBL signatures
score CLAMAV_MBL 7.5

# ClamAV MSRBL signatures from http://www.msrbl.com/
meta CLAMAV_MSRBL (CLAMAV && __CLAMAV_MSRBL)
describe CLAMAV_MSRBL SPAM found by ClamAV MSRBL signatures
score CLAMAV_MSRBL 2.0

# ClamAV SecuriteInfo.com VX malware signatures from
# http://www.securiteinfo.com/services/clamav_unofficial_malwares_signatures.shtml
meta CLAMAV_VX (CLAMAV && __CLAMAV_VX)
describe CLAMAV_VX Malware found by SecuriteInfo.com VX signatures
score CLAMAV_VX 5.0
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

thomas10
Normal user
Normal user
Posts: 54
Joined: 2013-10-30 03:13

Re: SpamAssassin Score + hMailServer Score

Post by thomas10 » 2018-01-19 05:30

mattg wrote:Here is a snippet of my custom.cf
(I changed the name so that it wasn't overwritten on SpamAssassin upgrade)

Code: Select all

#   Some shortcircuiting, if the plugin is enabled
# 
ifplugin Mail::SpamAssassin::Plugin::Shortcircuit
#
#   default: strongly-whitelisted mails are *really* whitelisted now, if the
#   shortcircuiting plugin is active, causing early exit to save CPU load.
#   Uncomment to turn this on
#
shortcircuit USER_IN_WHITELIST       on
# shortcircuit USER_IN_DEF_WHITELIST   on
shortcircuit USER_IN_ALL_SPAM_TO     on
shortcircuit SUBJECT_IN_WHITELIST    on

#   the opposite; blacklisted mails can also save CPU
#
shortcircuit USER_IN_BLACKLIST       on
# shortcircuit USER_IN_BLACKLIST_TO    on
# shortcircuit SUBJECT_IN_BLACKLIST    on

#   if you have taken the time to correctly specify your "trusted_networks",
#   this is another good way to save CPU
#

endif # Mail::SpamAssassin::Plugin::Shortcircuit


# don't score URIBL
score URIBL_BLACK 0
score URIBL_RED 0
score URIBL_GREY 0
score URIBL_BLOCKED 0

# DNSBL scores
score URIBL_DBL_SPAM 4
score RCVD_IN_SBL 3


# blacklist from
blacklist_from *.top
blacklist_from *.eu
blacklist_from *.download
blacklist_from *.accountant
blacklist_from *.cf
blacklist_from *.party
blacklist_from *.review
blacklist_from *.faith
blacklist_from *.win
blacklist_from *.trade
blacklist_from *.webcam
blacklist_from *.racing
blacklist_from *.date
blacklist_from *.bid
blacklist_from *.cricket

# whitelist from
whitelist_from *@important_domain.com.au


## BELOW is my ClamAV Integration
## NOT needed for what you are doing


loadplugin ClamAV clamav.pm
full CLAMAV eval:check_clamav()
describe CLAMAV Clam AntiVirus detected something...
score CLAMAV 0.001

# Look for specific types of ClamAV detections
header __CLAMAV_PHISH X-Spam-Virus =~ /Yes.{1,30}Phishing/i
header __CLAMAV_PHISH_HEUR X-Spam-Virus =~ /Yes.{1,30}Phishing\.Heuristics\.Email/
header __CLAMAV_SANE X-Spam-Virus =~ /Yes.{1,30}Sanesecurity/i
header __CLAMAV_MBL X-Spam-Virus =~ /Yes.{1,30}MBL/
header __CLAMAV_MSRBL X-Spam-Virus =~ /Yes.{1,30}MSRBL/
header __CLAMAV_VX X-Spam-Virus =~ /Yes.{1,30}VX\./

# Give the above rules a very late priority so that they can see the output
# of previous rules - otherwise they don't work! Not sure what the correct
# priority should be but this seems to work...
priority __CLAMAV_PHISH 9999
priority __CLAMAV_PHISH_HEUR 9999
priority __CLAMAV_SANE 9999
priority __CLAMAV_MBL 9999
priority __CLAMAV_MSRBL 9999
priority __CLAMAV_VX 9999


# Work out what ClamAV detected and score accordingly

# ClamAV general signatures
meta CLAMAV_VIRUS (CLAMAV && !__CLAMAV_PHISH && !__CLAMAV_SANE && !__CLAMAV_MBL && !__CLAMAV_MSRBL && !__CLAMAV_VX)
describe CLAMAV_VIRUS Virus found by ClamAV default signatures
score CLAMAV_VIRUS 20.0

# ClamAV phishing signatures
meta CLAMAV_PHISH (CLAMAV && __CLAMAV_PHISH && !__CLAMAV_SANE && !__CLAMAV_PHISH_HEUR)
describe CLAMAV_PHISH Phishing email found by ClamAV default signatures
score CLAMAV_PHISH 10.0

# ClamAV phishing with heuristic engine (not signatures based, may lead to false positives)
# Available since ClamAV 0.91
meta CLAMAV_PHISH_HEUR (CLAMAV && __CLAMAV_PHISH_HEUR)
describe CLAMAV_PHISH_HEUR Phishing email found by ClamAV heuristic engine
score CLAMAV_PHISH_HEUR 2.0

# ClamAV SaneSecurity signatures from http://www.sanesecurity.com/clamav/
meta CLAMAV_SANE (CLAMAV && __CLAMAV_SANE)
describe CLAMAV_SANE SPAM found by ClamAV SaneSecurity signatures
score CLAMAV_SANE 15

# ClamAV MBL signatures from http://www.malware.com.br/
meta CLAMAV_MBL (CLAMAV && __CLAMAV_MBL)
describe CLAMAV_MBL Malware found by ClamAV MBL signatures
score CLAMAV_MBL 7.5

# ClamAV MSRBL signatures from http://www.msrbl.com/
meta CLAMAV_MSRBL (CLAMAV && __CLAMAV_MSRBL)
describe CLAMAV_MSRBL SPAM found by ClamAV MSRBL signatures
score CLAMAV_MSRBL 2.0

# ClamAV SecuriteInfo.com VX malware signatures from
# http://www.securiteinfo.com/services/clamav_unofficial_malwares_signatures.shtml
meta CLAMAV_VX (CLAMAV && __CLAMAV_VX)
describe CLAMAV_VX Malware found by SecuriteInfo.com VX signatures
score CLAMAV_VX 5.0

Hi Matt,

I know it has already passed few months. I want to know on where is the location to put your custom.cf? Is it under user directory (~\.spamassasin)?
Is it ok to put the whitelist_from rules in local.cf?

User avatar
mattg
Moderator
Moderator
Posts: 20123
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: SpamAssassin Score + hMailServer Score

Post by mattg » 2018-01-19 09:08

Yep a few months, and I've changed since then

I have a whitelist.cf for just my whiteliest entries
I have the KAM rule set in KAM.cf >> https://www.pccc.com/downloads/SpamAssassin/contrib/
I have non-KAM rules from same source
I have a zzLast.cf to negate any rules that auto created from the above two lists
I have a blacklist.cf, a matt.cf, a nerds.cf (does the country of origin stuff) and more.

Seems you can have multiple .cf files, and they ALL get read individually

All of these are in my /etc/spamassassin/ folder on my UBUNTU system. I don't use the Jam Software windows variant of SpamAssassin
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
jimimaseye
Moderator
Moderator
Posts: 8125
Joined: 2011-09-08 17:48

Re: SpamAssassin Score + hMailServer Score

Post by jimimaseye » 2018-01-19 10:00

mattg wrote: All of these are in my /etc/spamassassin/ folder on my UBUNTU system. I don't use the Jam Software windows variant of SpamAssassin
Its similar on the windows version. In JAM the local.cf is found in (default) C:\Program Files\JAM Software\SpamAssassin for Windows\etc\spamassassin. Place other custom .CFs here too.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
SorenR
Senior user
Senior user
Posts: 3184
Joined: 2006-08-21 15:38
Location: Denmark

Re: SpamAssassin Score + hMailServer Score

Post by SorenR » 2018-01-19 23:18

jimimaseye wrote:
mattg wrote: All of these are in my /etc/spamassassin/ folder on my UBUNTU system. I don't use the Jam Software windows variant of SpamAssassin
Its similar on the windows version. In JAM the local.cf is found in (default) C:\Program Files\JAM Software\SpamAssassin for Windows\etc\spamassassin. Place other custom .CFs here too.
The file path.config in the SpamAssassin directory will specify locations..

Code: Select all

DEF_RULES_DIR=./share/spamassassin
LOCAL_RULES_DIR=./etc/spamassassin
LOCAL_STATE_DIR=./share
SørenR.

“With age comes wisdom, but sometimes age comes alone.”
- Oscar Wilde

Post Reply