How to set score for certain texts?

Use this forum for discussions about SpamAssassin and anti-spam in general.
Post Reply
thomas10
Normal user
Normal user
Posts: 132
Joined: 2013-10-30 03:13

How to set score for certain texts?

Post by thomas10 » 2020-05-22 04:30

Hi All,

I recently receive some email sent from different email address and IP but with same email message.
I know that it can be resolved by decreasing the delete threshold but I haven't tried on how to set score for certain texts. So I need to learn and get help from you guys on this.
- May I know the file location that set scores for texts? Under "Share" folder?
- How to set score for texts like like bitcoin, darknet, etc?

Below is the email message.
Good day

I run a website in the darknet, I produce all kinds of services - basically it is destruction to property and harm. above all, all but the killings. Often this happens because of rejected love or competition at work. This month he talked me and set me the mission of splashing acid in your face. Default task - quickly, hurts, forever. Without too much fuss. I get receive only after doing the order. So, now I propose you send money to me to be inactive, I suggest this to nearly all the victims. If I do not see money from you, then my person will fulfill the mission. If you transfer me money, besides to my inaction, I will give you the info that I have about the customer. After completing the task, I always lose the performer, so I have an option, to get $1400 from you for information about the customer and my inaction, or to receive $ 4000 from the customer, but with a high probability of losing the performer.

I’m getting paid in BTC, its my Bitcoin address - 126Sciww6nR5FSUr9AqasqoE8mjZ2Lhm45
The summary I told above.
24 hours to decide and pay.

Kendo
Normal user
Normal user
Posts: 135
Joined: 2015-07-08 23:33
Location: Rural Australia

Re: How to set score for certain texts?

Post by Kendo » 2020-05-22 05:07

There is a word filter that can do this. See https://www.hmailserver.com/forum/viewt ... =7&t=34421

User avatar
jimimaseye
Moderator
Moderator
Posts: 10060
Joined: 2011-09-08 17:48

Re: How to set score for certain texts?

Post by jimimaseye » 2020-05-22 09:59

I'm confident that defaults defaults spamassassin ruleset will have identified this as spam already and that your score threshold is too high. It is the easiest thing to do without repercussions.

What are your settings? run this and post the results: https://www.hmailserver.com/forum/viewt ... 20&t=30914

Also post the headers of the message as it was received.

[Entered by mobile. Excuse my spelling.]
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

thomas10
Normal user
Normal user
Posts: 132
Joined: 2013-10-30 03:13

Re: How to set score for certain texts?

Post by thomas10 » 2020-07-15 11:19

jimimaseye wrote:
2020-05-22 09:59
I'm confident that defaults defaults spamassassin ruleset will have identified this as spam already and that your score threshold is too high. It is the easiest thing to do without repercussions.

What are your settings? run this and post the results: https://www.hmailserver.com/forum/viewt ... 20&t=30914

Also post the headers of the message as it was received.

[Entered by mobile. Excuse my spelling.]
Hi Jimi, It's me again. I tend to create a new topic but I found that it is very similar to my previous topic here. So I'm gonna state here first (Sorry if this is inappropriate, I will open a new topic if you think so) :oops:

The reason I reply your message here is because yes, you are right. The score for delete threshold is high but at least SA is still able to capture it and score it accordingly. (I can't set it too low due to there are still false positives happening)
But I found that some spam mails are able to get low score on SA/ bypass SA. That's the reason why I created a forum earlier to ask about setting Global rules using Regex to capture certain texts under Subject and send to my email.

Now I am trying to set the rule under SA so the texts can score very high once matched.

I have never done a score rule for texts in SA before, so I'm here to learn.
I have read your topic viewtopic.php?f=7&t=26661&hilit=spamassassin&start=60
I think to add below in the local.cf according to your sample.

Code: Select all

header PHISH_SUBJECT_1  Subject =~  /^.*(Mailbox is full|Release Email|exceeded its limit|almost reached  limit).*$/i
header PHISH_SUBJECT_2  Subject =~  /^.*(Server Notification|free space|New Mails Has Been Blocked|Removal of Your Account).*$/i
meta PHISH_SUBJECT  (PHISH_SUBJECT_1 && PHISH_SUBJECT_2)
score  PHISH_SUBJECT 7.0
describe PHISH_SUBJECT		My Phishing Rule
Q1, If I put this code in local.cf, does it mean that SA will capture emails and set score 7 if the subject contains "Release Emailss", "dexceeded its limit", "free spaces", and others related texts stated?

Q2, If I want to add header PHISH_SUBJECT_3, can I set the meta as below?
meta PHISH_SUBJECT (PHISH_SUBJECT_1 && PHISH_SUBJECT_2 && PHISH_SUBJECT_3)
Q3, Does it have a text length limit for each Regex? Because I met the length error when set Global rules in Hmail using Regex. :shock:

User avatar
RvdH
Senior user
Senior user
Posts: 3235
Joined: 2008-06-27 14:42
Location: The Netherlands

Re: How to set score for certain texts?

Post by RvdH » 2020-07-15 15:16

Q1) Rule looks fine and should be captured when in local.cf

Q2) meta PHISH_SUBJECT (PHISH_SUBJECT_1 && PHISH_SUBJECT_2 && PHISH_SUBJECT_3) would work if you specify PHISH_SUBJECT_3 regex

Q3) I haven't heard/seen limits of spamassassin regex's
CIDR to RegEx: d-fault.nl/cidrtoregex
DNS Lookup: d-fault.nl/dnstools
DKIM Generator: d-fault.nl/dkimgenerator
DNSBL Lookup: d-fault.nl/dnsbllookup
GEOIP Lookup: d-fault.nl/geoiplookup

User avatar
jimimaseye
Moderator
Moderator
Posts: 10060
Joined: 2011-09-08 17:48

Re: How to set score for certain texts?

Post by jimimaseye » 2020-07-15 18:33

This will help you: https://cwiki.apache.org/confluence/dis ... itingRules

BTW: using && means 'AND'

so (PHISH_SUBJECT_1 && PHISH_SUBJECT_2)

means that both SUBJECT_1 AND SUBJECT_2 must both have matches to acheive overall rule score (which I dont think is what you want. Dont you want an "OR" instead of 'And' ?)
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

thomas10
Normal user
Normal user
Posts: 132
Joined: 2013-10-30 03:13

Re: How to set score for certain texts?

Post by thomas10 » 2020-07-16 03:47

RvdH wrote:
2020-07-15 15:16
Q1) Rule looks fine and should be captured when in local.cf

Q2) meta PHISH_SUBJECT (PHISH_SUBJECT_1 && PHISH_SUBJECT_2 && PHISH_SUBJECT_3) would work if you specify PHISH_SUBJECT_3 regex

Q3) I haven't heard/seen limits of spamassassin regex's
Thanks so much RvDH for your answer. Will compile them up and set them up in local.cf once ready.

jimimaseye wrote:
2020-07-15 18:33
This will help you: https://cwiki.apache.org/confluence/dis ... itingRules

BTW: using && means 'AND'

so (PHISH_SUBJECT_1 && PHISH_SUBJECT_2)

means that both SUBJECT_1 AND SUBJECT_2 must both have matches to acheive overall rule score (which I dont think is what you want. Dont you want an "OR" instead of 'And' ?)
You are right again on it. I would like to set one rule that has few phish subjects together with same score in future (Phish Subject1,2,3...).
So the condiition will be as below.
if contains phish subject 1 or 2 or 3, then set score 7.0
Thus, I believe OR will be more suitable since I don't want to mix them up. Thanks Jimi.
meta PHISH_SUBJECT (PHISH_SUBJECT_1 OR PHISH_SUBJECT_2 OR PHISH_SUBJECT_3)

But one more question guys, lets say if I add the word "bitcoin" and set score to 10, and found that in the SA update, there is also a rule set score 9 for the word "bitcoin", what will happen next?

User avatar
jimimaseye
Moderator
Moderator
Posts: 10060
Joined: 2011-09-08 17:48

Re: How to set score for certain texts?

Post by jimimaseye » 2020-07-16 09:02

The link i posted should tell you how to do OR comparisons.
thomas10 wrote:
2020-07-16 03:47
But one more question guys, lets say if I add the word "bitcoin" and set score to 10, and found that in the SA update, there is also a rule set score 9 for the word "bitcoin", what will happen next?
You'll get 19.

[Entered by mobile. Excuse my spelling.]
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
RvdH
Senior user
Senior user
Posts: 3235
Joined: 2008-06-27 14:42
Location: The Netherlands

Re: How to set score for certain texts?

Post by RvdH » 2020-07-17 04:09

Or = ||
CIDR to RegEx: d-fault.nl/cidrtoregex
DNS Lookup: d-fault.nl/dnstools
DKIM Generator: d-fault.nl/dkimgenerator
DNSBL Lookup: d-fault.nl/dnsbllookup
GEOIP Lookup: d-fault.nl/geoiplookup

thomas10
Normal user
Normal user
Posts: 132
Joined: 2013-10-30 03:13

Re: How to set score for certain texts?

Post by thomas10 » 2020-07-17 06:40

jimimaseye wrote:
2020-07-16 09:02
The link i posted should tell you how to do OR comparisons.
After read through, I only manage to find && and +. For the OR, I believe will still be the || that I set previously. RvDH also provide the answer for it too. Thanks again. I have bookmarked the page for future reference.
jimimaseye wrote:
2020-07-16 09:02
thomas10 wrote:
2020-07-16 03:47
But one more question guys, lets say if I add the word "bitcoin" and set score to 10, and found that in the SA update, there is also a rule set score 9 for the word "bitcoin", what will happen next?
You'll get 19.

[Entered by mobile. Excuse my spelling.]
Ahh, so it will add up all the score. I see.
RvdH wrote:
2020-07-17 04:09
Or = ||
I was thinking about it too. I thought the || is not applicable under meta there. Gonna try it. :D


Hmm, it seems that I have all the answer that I need so far. Thanks again guys. Appreciated.

thomas10
Normal user
Normal user
Posts: 132
Joined: 2013-10-30 03:13

Re: How to set score for certain texts?

Post by thomas10 » 2020-07-30 12:03

@jimimaseye

I have faced an issue.
I tried to set score for the text "upgrαde required" in SA but when save the local cf, a pop up said the file contains characters in Unicode format which will be lost if save the file as ANSI encoded text file.

The culprit is the letter "a" in the text upgrade. Please refer attached picture.
Will the local cf file still work if change the Encoding to other than ANSI?
Attachments
Untitled.jpg

User avatar
jimimaseye
Moderator
Moderator
Posts: 10060
Joined: 2011-09-08 17:48

Re: How to set score for certain texts?

Post by jimimaseye » 2020-07-30 15:25

lookup "Spamassassin Plugin::ReplaceTags" in Google.

This section of my local.cf cheks for various methods of entering the words "iTunes", "Amazon", "Account", "Customer", "Verify", "Please update" (and various combinations thereof).

Code: Select all

rawbody  __MY_PHISH_CIRCUMVENT_ATTEMPT1   /(i&\#932;unes|arnazon|&\#945;ccount|p&\#945;|&\#959;|&\#957;)/i
header   __MY_PHISH_CIRCUMVENT_ATTEMPT2   Subject:raw =~ /(=F0|=C3=BElease|(u=C3=BE|=B5(=C3=BE|p))dat|=C3=ACnf|a((cco=b5|a=C4=8B=C4=8Bo(u|=B5))nt|=C4=8B=C4=8Bess))/i

# Next rule relies on 25_replace.cf being present
ifplugin Mail::SpamAssassin::Plugin::ReplaceTags
body __MY_PHISH_CIRCUMVENT_ATTEMPT3 	/(?!account)<A><C><C><O><U><N><T>|(?!customer)<C><U><S><T><O><M><E><R>|(?!verif(y|i))<V><E><R><I><F>(<Y>|<I>)/i
replace_rules __MY_PHISH_CIRCUMVENT_ATTEMPT3
endif   # Mail::SpamAssassin::Plugin::ReplaceTags

meta     MY_PHISH_CIRCUMVENT_ATTEMPT      (__MY_PHISH_CIRCUMVENT_ATTEMPT1 || __MY_PHISH_CIRCUMVENT_ATTEMPT2 || __MY_PHISH_CIRCUMVENT_ATTEMPT3)
score    MY_PHISH_CIRCUMVENT_ATTEMPT 7.0
describe MY_PHISH_CIRCUMVENT_ATTEMPT Deliberate fooling with encoding guaranteed dodgy
It will help.
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

thomas10
Normal user
Normal user
Posts: 132
Joined: 2013-10-30 03:13

Re: How to set score for certain texts?

Post by thomas10 » 2020-10-09 05:34

@jimimaseye

Done with rules for the subject and test working. But how about the message body?
I have done with some testing with the rules below, but it doesn't seems to be working.
I insert texts below in the message body of the test message via external email address, but the message still goes through to my email. Is there anything wrong with the code?

Code: Select all

body PHISH_BODY_1  /^.*([b]cancel de-activation|cancel deactivation[/b]).*$/i
body PHISH_BODY_2  /^.*([b]Keep Your Old Password[/b]).*$/i
meta PHISH_BODY  (PHISH_BODY_1|PHISH_BODY_2)
score  PHISH_BODY 10.0
describe PHISH_BODY		My Phishing BODY Rule

User avatar
jimimaseye
Moderator
Moderator
Posts: 10060
Joined: 2011-09-08 17:48

Re: How to set score for certain texts?

Post by jimimaseye » 2020-10-09 09:33

thomas10 wrote:
2020-10-09 05:34

body PHISH_BODY_1 /^.*([ b]cancel de-activation|cancel deactivation[ /b]).*$/i
body PHISH_BODY_2 /^.*([ b]Keep Your Old Password[ /b]).*$/i
meta PHISH_BODY (PHISH_BODY_1|PHISH_BODY_2)
score PHISH_BODY 10.0
describe PHISH_BODY My Phishing BODY Rule[/code]

Why do you have [ b] tags there? It will not work if you have those.

(And you can use the 'OR' to separate different values in one single rule - you dont need to be using meta rules in the example you have given. My examples should be enough to show you.)

Code: Select all

body PHISH_BODY       /(cancel de-activation|cancel deactivation|Keep Your Old Password)/i
score  PHISH_BODY      10.0
describe PHISH_BODY   My Phishing BODY Rule
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

thomas10
Normal user
Normal user
Posts: 132
Joined: 2013-10-30 03:13

Re: How to set score for certain texts?

Post by thomas10 » 2020-10-09 09:53

jimimaseye wrote:
2020-10-09 09:33
thomas10 wrote:
2020-10-09 05:34

body PHISH_BODY_1 /^.*([ b]cancel de-activation|cancel deactivation[ /b]).*$/i
body PHISH_BODY_2 /^.*([ b]Keep Your Old Password[ /b]).*$/i
meta PHISH_BODY (PHISH_BODY_1|PHISH_BODY_2)
score PHISH_BODY 10.0
describe PHISH_BODY My Phishing BODY Rule[/code]

Why do you have [ b] tags there? It will not work if you have those.

(And you can use the 'OR' to separate different values in one single rule - you dont need to be using meta rules in the example you have given. My examples should be enough to show you.)

Code: Select all

body PHISH_BODY       /(cancel de-activation|cancel deactivation|Keep Your Old Password)/i
score  PHISH_BODY      10.0
describe PHISH_BODY   My Phishing BODY Rule

Hi jimi, Apologies with the “b”, I put it when I was about to type the code at here to make them look bold.
On the actual code, there is no "b". Sorry to confuse you.

Ok, will use your sample. I input them in the local.cf. I believe that restart SA service every time updating the code will be the best to make sure the code run normally. (I always did that when updating the blacklist and whitelist cf):lol:


Aside on this, I found that in the hmail log file, there are a lot of different IPs trying to access but most of them get "Authentication failed. Restarting authentication process."
These IPs were trying to login as xxxx@[our domain].com. I have tried to block them via IP range. I know they have failed to authenticate but my log file size is increased due to the repeating trying by those IPs.
Is there any other way to overcome this issue? What is the possible reason that cause it to happen in your point of view?

User avatar
katip
Senior user
Senior user
Posts: 1161
Joined: 2006-12-22 07:58
Location: Istanbul

Re: How to set score for certain texts?

Post by katip » 2020-10-09 10:43

thomas10 wrote:
2020-10-09 09:53
Ok, will use your sample. I input them in the local.cf. I believe that restart SA service every time updating the code will be the best to make sure the code run normally. (I always did that when updating the blacklist and whitelist cf):lol:
assuming your SA is JAM Win:
there should be a file in <windows>\System32\config\systemprofile\.spamassassin\user_prefs
changes there apply instantly. no need to restart SA.
Katip
--
HMS 5.7, MariaDB 10.4.10, SA 4.0.0, ClamAV 0.103.8

thomas10
Normal user
Normal user
Posts: 132
Joined: 2013-10-30 03:13

Re: How to set score for certain texts?

Post by thomas10 » 2020-10-09 10:57

katip wrote:
2020-10-09 10:43
thomas10 wrote:
2020-10-09 09:53
Ok, will use your sample. I input them in the local.cf. I believe that restart SA service every time updating the code will be the best to make sure the code run normally. (I always did that when updating the blacklist and whitelist cf):lol:
assuming your SA is JAM Win:
there should be a file in <windows>\System32\config\systemprofile\.spamassassin\user_prefs
changes there apply instantly. no need to restart SA.
Huh, That is fresh news to me. I just checked on the location at C:\Windows\System32\config\systemprofile\.spamassassin, there are only files with bayes journal, toks, etc, no user_prefs file inside.

User avatar
katip
Senior user
Senior user
Posts: 1161
Joined: 2006-12-22 07:58
Location: Istanbul

Re: How to set score for certain texts?

Post by katip » 2020-10-09 11:24

thomas10 wrote:
2020-10-09 10:57
Huh, That is fresh news to me. I just checked on the location at C:\Windows\System32\config\systemprofile\.spamassassin, there are only files with bayes journal, toks, etc, no user_prefs file inside.
then put one and try.
(no extension, filename as only user_prefs)
Katip
--
HMS 5.7, MariaDB 10.4.10, SA 4.0.0, ClamAV 0.103.8

thomas10
Normal user
Normal user
Posts: 132
Joined: 2013-10-30 03:13

Re: How to set score for certain texts?

Post by thomas10 » 2020-10-09 11:34

katip wrote:
2020-10-09 11:24
thomas10 wrote:
2020-10-09 10:57
Huh, That is fresh news to me. I just checked on the location at C:\Windows\System32\config\systemprofile\.spamassassin, there are only files with bayes journal, toks, etc, no user_prefs file inside.
then put one and try.
(no extension, filename as only user_prefs)
Ok, If I put there, do I still need to change the setting on both local.cf and user_prefs file?

User avatar
katip
Senior user
Senior user
Posts: 1161
Joined: 2006-12-22 07:58
Location: Istanbul

Re: How to set score for certain texts?

Post by katip » 2020-10-09 15:09

thomas10 wrote:
2020-10-09 11:34
Ok, If I put there, do I still need to change the setting on both local.cf and user_prefs file?
not neccesarily. however i use it for urgent cases. later i transfer changes to relevant .cf files and delete from user_prefs the next day. (SA anyway restarts daily around midnight)
some critical things which i play frequently with stay in user_prefs though. YMMV
Katip
--
HMS 5.7, MariaDB 10.4.10, SA 4.0.0, ClamAV 0.103.8

User avatar
mattg
Moderator
Moderator
Posts: 22437
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: How to set score for certain texts?

Post by mattg » 2020-10-10 01:28

thomas10 wrote:
2020-10-09 09:53
Aside on this, I found that in the hmail log file, there are a lot of different IPs trying to access but most of them get "Authentication failed. Restarting authentication process."
Many of us block AUTH on port 25 to mitigate these threats.

(This is done in the hMailserverv.ini, and MAY require that your regular mail clients need to connect on another port. Consider the consequences before you implement)

Even with Port 25 blocked for AUTH, I get an average of more than 15 attempts per day on other mail ports where hackers are trying to guess login credentials. Use strong passwords
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

thomas10
Normal user
Normal user
Posts: 132
Joined: 2013-10-30 03:13

Re: How to set score for certain texts?

Post by thomas10 » 2020-10-13 04:23

mattg wrote:
2020-10-10 01:28
thomas10 wrote:
2020-10-09 09:53
Aside on this, I found that in the hmail log file, there are a lot of different IPs trying to access but most of them get "Authentication failed. Restarting authentication process."
Many of us block AUTH on port 25 to mitigate these threats.

(This is done in the hMailserverv.ini, and MAY require that your regular mail clients need to connect on another port. Consider the consequences before you implement)

Even with Port 25 blocked for AUTH, I get an average of more than 15 attempts per day on other mail ports where hackers are trying to guess login credentials. Use strong passwords
You are right about it. Port 25 is the one these IPs are accessing. They either tried with current email address or random email address with our domain name.

We use port 25 also, so don't think block it will be a good way for us. Seems like blocking these IPs via IP Range will be the only choice to go for then. Hate these hackers. :evil:

User avatar
mattg
Moderator
Moderator
Posts: 22437
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: How to set score for certain texts?

Post by mattg » 2020-10-13 07:51

You can't completely block port 25 - all incoming mail comes via port 25

What we block is the ability to use Port 25 to AUTH

anyone who sends mail on my server must AUTH, and they must AUTH on another port eg 465 or 587, both of which are secured with a suitable certificate
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

Post Reply