SpamAssassin Bootcamp (sa-learn) train BAYES

This section contains scripts that hMailServer has contributed with. hMailServer 5 is needed to use these.
User avatar
jimimaseye
Moderator
Moderator
Posts: 8175
Joined: 2011-09-08 17:48

Re: SpamAssassin Bootcamp (sa-learn) train BAYES

Post by jimimaseye » 2018-08-27 11:34

Whilst your tinkering, is this any interest to you: viewtopic.php?f=20&t=28052 ?
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

palinka
Senior user
Senior user
Posts: 1303
Joined: 2017-09-12 17:57

Re: SpamAssassin Bootcamp (sa-learn) train BAYES

Post by palinka » 2018-08-27 12:46

jimimaseye wrote:
2018-08-27 11:34
Whilst your tinkering, is this any interest to you: viewtopic.php?f=20&t=28052 ?
That's very cool but I already set up an account to which all spam including spam above the delete threshold gets sent. That way I can sort it for learning purposes because I know that no users are sorting. At least they're not sorting very in any consistent manner. :roll:

rub.ak
New user
New user
Posts: 2
Joined: 2018-09-22 19:44

Re: SpamAssassin Bootcamp (sa-learn) train BAYES

Post by rub.ak » 2018-09-22 19:49

I used sa-learn.0.6.1.rar and encountered the problem of copying mail (spam.cmd ham.cmd)

Code: Select all

C:\SpamAssassin\temp>COPY "C:\Program Files (x86)\hMailServer\Data\home.aln\spam
\4B\{4BEF5CA4-5DBD-4668-B8B2-3C8104BD65BB}.eml" C:\SpamAssassin\temp\ham\5585.em
l /Y
Системе не удается найти указанный путь.
Скопировано файлов:         0.
How to solve the problem with copying?

User avatar
jimimaseye
Moderator
Moderator
Posts: 8175
Joined: 2011-09-08 17:48

Re: SpamAssassin Bootcamp (sa-learn) train BAYES

Post by jimimaseye » 2018-09-22 20:46

rub.ak wrote:
2018-09-22 19:49
I used sa-learn.0.6.1.rar and encountered the problem of copying mail (spam.cmd ham.cmd)

Code: Select all

C:\SpamAssassin\temp>COPY "C:\Program Files (x86)\hMailServer\Data\home.aln\spam
\4B\{4BEF5CA4-5DBD-4668-B8B2-3C8104BD65BB}.eml" C:\SpamAssassin\temp\ham\5585.em
l /Y
Системе не удается найти указанный путь.
Скопировано файлов:         0.
How to solve the problem with copying?

What version of hns do you have? Can you run this and post the results: viewtopic.php?f=20&t=30914
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

rub.ak
New user
New user
Posts: 2
Joined: 2018-09-22 19:44

Re: SpamAssassin Bootcamp (sa-learn) train BAYES

Post by rub.ak » 2018-09-22 22:32

jimimaseye wrote:
2018-09-22 20:46
Thanks for the response.
I solved the problem myself, it was enough to create the folders / temp / spam, / temp / ham

User avatar
jimimaseye
Moderator
Moderator
Posts: 8175
Joined: 2011-09-08 17:48

Re: SpamAssassin Bootcamp (sa-learn) train BAYES

Post by jimimaseye » 2018-09-22 22:54

Ok
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

ashtec014
Normal user
Normal user
Posts: 41
Joined: 2019-09-05 11:56

Re: SpamAssassin Bootcamp (sa-learn) train BAYES

Post by ashtec014 » 2019-11-28 11:55

SorenR wrote:
2014-08-09 13:31
The success of SpamAssassin relies on a well trained Bayes database. There are many ways to train your Bayes database, this is my shot at doing it.
NB! wrote:How to obtain SpamAssassin and the installation and configuration of SpamAssassin is NOT described here! Search elsewhere on this forum to obtain this information.
The idea came from my play/toy MailServer (Postfix, Dovecut & MailScanner) on my Synology DS209+II NAS. There it is basically a no-brainer to set up as everything works off of MailDirs.

So how to do ... on a hMailServer 5.4.2-B1964 system ...

1 -- Build a script using the COM api to find and extract relevant emails and what could be more natural than to assume INBOX is good and SPAM is bad. Also, if HAM end up in SPAM (or visa versa) you move it to the respective folder (INBOX or SPAM) and at the next scheduled run, the email will be classified differently. I execute this script using Windows Schedule at 04:00 in the morning when everyone is (supposed to be) at sleep.

2 -- A global rule in hMailServer to move emails tagged as SPAM into a SPAM folder. Setting this rule as a global rule will ensure that ALL users of hMailServer are covered. If the SPAM folder do not exist, it will be created by hMailServer automatically.

Rule name: sa-learn

Code: Select all

Criteria -> Custom header field -> X-hMailServer-Spam = YES
Action -> Move to IMAP folder -> IMAP folder = SPAM
Some admins may want to monitor what is tagged as SPAM, like me, so I forward a copy to spam@my-domain.tld with this revised global rule.

Rule name: sa-learn (BigBrother version) :: Use AND

Code: Select all

Criteria -> Custom header field -> X-hMailServer-Spam = YES
Criteria -> Custom header field -> X-hMailServer-LoopCount < 1
Action -> Move to IMAP folder -> IMAP folder = SPAM
Action -> Forward email -> To = spam@my-domain.tld
"X-hMailServer-LoopCount" is used to prevent loops. By checking for value < 1 we make sure it is only run once. Thus SPAM will stay in the spam@my-domain.tld INBOX.

3 -- Now that we have both good and bad emails defined, we need to pull them off of the server. For that I have choosen VBScript to interact with the hMailServer COM API.
Functional Description: wrote: The VBScript (sa-learn.vbs) will work with ONE domain at present as I only have one domain. Using the COM API it will locate and process all account addresses for that domain - except those addresses listed in the exception list.

The script will do two passes, one for INBOX and one for SPAM, and generate two .cmd files (HAMCopy.cmd & SPAMCopy.cmd) to be run by the script.

During the two passes, the number of messages in the respective folders are checked and only the last (max.) 20 messages are processed. This procedure is based on the assumption that the COM API will return data sorted by table ID and from examining the database it appears that hMailServer simply adds new message ID's to the table in favor of reusing deleted message ID's.

HAMCopy.cmd and SPAMCopy.cmd simply copies the selected .eml files to a HAM or SPAM directory.

A third .cmd file (sa-learn.cmd) is also executed by the script and this .cmd file contains the commands to execute sa-learn --spam, sa-learn --ham, sa-learn --sync and sa-learn --backup as it is customary on Unix type systems.

On a Dual-Core 3GHz, 4GB RAM, SATA, System w/ Windows Server 2003R2 it took almost 20 minutes to process ~4.200 HAM and ~5.600 SPAM emails.
sa-learn.vbs

Code: Select all

Option Explicit
   '
   ' Version 0.1.0 09-08-2014, Soren Rathje - Initial version.
   '

   Dim hmAdmin, hmPassword, hmDomain, hmSPAMFolder, hmSPAMDir, hmHAMFolder, hmHAMDir, hmExcludeAddress
   Dim i, j, s, objApp, objDomain, objAccount, objIMAPFolder, objMessage
   Dim fsoSPAM, fsoHAM, fsoSALearn, objFSO, objSPAM, objHAM, objShell

   '
   ' Configuration parameters - BEGIN
   '
   hmAdmin = "Administrator"          ' hMailServer Administrator user
   hmPassword = "********"            ' hMailServer Administrator password
   hmDomain = "my-domain.tld"         ' Domain name
   hmSPAMFolder = "SPAM"              ' SPAM IMAP folder
   hmSPAMDir = "C:\hMailServer\SPAM"  ' You need to create this directory!
   hmHAMFolder = "INBOX"              ' HAM IMAP Folder
   hmHAMDir = "C:\hMailServer\HAM"    ' You need to create this directory!
   hmExcludeAddress = "spam@my-domain.tld, surveillance@my-domain.tld"

   fsoSPAM = "C:\hMailServer\Events\SPAMCopy.cmd"
   fsoHAM = "C:\hMailServer\Events\HAMCopy.cmd"
   fsoSALearn = "C:\hMailServer\Events\sa-learn.cmd"
   '
   ' Configuration parameters - END
   '

   Set objShell = WScript.CreateObject("WScript.Shell")
   Set objFSO = CreateObject("Scripting.FileSystemObject")
   Set objApp = CreateObject("hMailServer.Application")
   Call objApp.Authenticate(hmAdmin, hmPassword)
   Set objDomain = objApp.Domains.ItemByName(hmDomain)

   '
   ' Find SPAM messages
   '
   Set objSPAM = objFSO.CreateTextFile(fsoSPAM,True)
   For i = 0 to objDomain.Accounts.Count -1
      Set objAccount = objDomain.Accounts.Item(i)

      ' DO NOT process excluded and non-active accounts.
      If (NOT InStr(hmExcludeAddress, objAccount.Address)) * objAccount.Active Then

         Set objIMAPFolder = objAccount.IMAPFolders.ItemByName(hmSPAMFolder)

         ' If no messages - skip
         If objIMAPFolder.Messages.Count > 0 Then

            s = 0
            If objIMAPFolder.Messages.Count - 20 > 0 Then s = objIMAPFolder.Messages.Count - 20

            For j = s to objIMAPFolder.Messages.Count -1
               Set objMessage = objIMAPFolder.Messages.Item(j)
               objSPAM.Write "COPY " & objMessage.FileName & " " & hmSPAMDir & " /Y" & vbCrLf
            Next
         End If
      End If
   Next
   objSPAM.Close

   '
   ' Find HAM messages
   '
   Set objHAM = objFSO.CreateTextFile(fsoHAM,True)
   For i = 0 to objDomain.Accounts.Count -1
      Set objAccount = objDomain.Accounts.Item(i)

      ' DO NOT process excluded and non-active accounts.
      If (NOT InStr(hmExcludeAddress, objAccount.Address)) * objAccount.Active Then

         Set objIMAPFolder = objAccount.IMAPFolders.ItemByName(hmHAMFolder)

         ' If no messages - skip
         If objIMAPFolder.Messages.Count > 0 Then

            s = 0
            If objIMAPFolder.Messages.Count - 20 > 0 Then s = objIMAPFolder.Messages.Count - 20

            For j = s to objIMAPFolder.Messages.Count -1
               Set objMessage = objIMAPFolder.Messages.Item(j)
               objHAM.Write "COPY " & objMessage.FileName & " " & hmHAMDir & " /Y" & vbCrLf
            Next
         End If
      End If
   Next
   objHAM.Close

   '
   ' Execute file copy and sa-learn.exe - sequentially - no StdOut.
   '
   objShell.Run fsoSPAM, 0, true
   objShell.Run fsoHAM, 0, true
   objShell.Run fsoSALearn, 0, true
sa-learn.cmd

Code: Select all

C:\SpamAssassin\sa-learn.exe --siteconfigpath="C:\SpamAssassin\etc\spamassassin" --dbpath "C:\Documents and Settings\Default User\.spamassassin\bayes" --spam "C:\hMailServer\SPAM\*.eml"
C:\SpamAssassin\sa-learn.exe --siteconfigpath="C:\SpamAssassin\etc\spamassassin" --dbpath "C:\Documents and Settings\Default User\.spamassassin\bayes" --ham "C:\hMailServer\HAM\*.eml"
C:\SpamAssassin\sa-learn.exe --siteconfigpath="C:\SpamAssassin\etc\spamassassin" --dbpath "C:\Documents and Settings\Default User\.spamassassin\bayes" --sync
C:\SpamAssassin\sa-learn.exe --siteconfigpath="C:\SpamAssassin\etc\spamassassin" --dbpath "C:\Documents and Settings\Default User\.spamassassin\bayes" --backup > "C:\Documents and Settings\Default User\.spamassassin\bayes_backup"
REM DELETE C:\hMailServer\SPAM\*.eml /Q
REM DELETE C:\hMailServer\HAM\*.eml /Q
Disclaimer: wrote: I take no responsibility for what you may or may not do with the above script/shell script. It works for me - it may not work for you. I DO NOT GUARANTEE THIS CODE TO BE BUG-FREE! USE AT YOUR OWN RISK! WHATEVER YOU DO - IT'S NOT MY FAULT!

AND remember; Real men do NOT backup - but they CRY a lot!

Please feel free to adopt and modify.
Hello,

I want to use this to my existing hmailserver installation, however I am a bit confused how to use and implement the script. Is there any step-by-step instruction (similar to what Jimimaseye did to this post https://www.hmailserver.com/forum/viewt ... 91#p174991 ) where to put these scripts? Which folder, etc.

Thank you.

User avatar
SorenR
Senior user
Senior user
Posts: 3228
Joined: 2006-08-21 15:38
Location: Denmark

Re: SpamAssassin Bootcamp (sa-learn) train BAYES

Post by SorenR » 2019-11-28 12:53

ashtec014 wrote:
2019-11-28 11:55
SorenR wrote:
2014-08-09 13:31
The success of SpamAssassin relies on a well trained Bayes database. There are many ways to train your Bayes database, this is my shot at doing it.
NB! wrote:How to obtain SpamAssassin and the installation and configuration of SpamAssassin is NOT described here! Search elsewhere on this forum to obtain this information.
The idea came from my play/toy MailServer (Postfix, Dovecut & MailScanner) on my Synology DS209+II NAS. There it is basically a no-brainer to set up as everything works off of MailDirs.

So how to do ... on a hMailServer 5.4.2-B1964 system ...

1 -- Build a script using the COM api to find and extract relevant emails and what could be more natural than to assume INBOX is good and SPAM is bad. Also, if HAM end up in SPAM (or visa versa) you move it to the respective folder (INBOX or SPAM) and at the next scheduled run, the email will be classified differently. I execute this script using Windows Schedule at 04:00 in the morning when everyone is (supposed to be) at sleep.

2 -- A global rule in hMailServer to move emails tagged as SPAM into a SPAM folder. Setting this rule as a global rule will ensure that ALL users of hMailServer are covered. If the SPAM folder do not exist, it will be created by hMailServer automatically.

bla bla bla
bla bla bla
Disclaimer: wrote: I take no responsibility for what you may or may not do with the above script/shell script. It works for me - it may not work for you. I DO NOT GUARANTEE THIS CODE TO BE BUG-FREE! USE AT YOUR OWN RISK! WHATEVER YOU DO - IT'S NOT MY FAULT!

AND remember; Real men do NOT backup - but they CRY a lot!

Please feel free to adopt and modify.
Hello,

I want to use this to my existing hmailserver installation, however I am a bit confused how to use and implement the script. Is there any step-by-step instruction (similar to what Jimimaseye did to this post https://www.hmailserver.com/forum/viewt ... 91#p174991 ) where to put these scripts? Which folder, etc.

Thank you.
Did you read the whole thread? The code in the very first post will probably not even work anymore after 5 years of Microsoft Updates. I do not have admin privs. on this board so code changes are posted as they are made.

Once you have read everything you will realize that it does not really matter where you put the scripts, they have hardcoded paths in them. I have sa-learn.vbs and sa-learn.cmd in my c:\spamassassin directory.
SørenR.

“With age comes wisdom, but sometimes age comes alone.”
- Oscar Wilde

ashtec014
Normal user
Normal user
Posts: 41
Joined: 2019-09-05 11:56

Re: SpamAssassin Bootcamp (sa-learn) train BAYES

Post by ashtec014 » 2019-12-11 18:24

SorenR wrote:
2018-07-28 13:43
Ok... Two steps forward and one step back...

I suspect there continues to be an issue with curly brackets.

Version 0.6.1 is back to creating HAM.CMD and SPAM.CMD that will copy mails to c:\spamassassin\temp\ham and c:\spamassassin\temp\spam. SA-LEARN.CMD does the actual learning.

DO NOT forget to delete the HAM and SPAM files in .\temp or you WILL get the permission issue. You need to recreate the HAM and SPAM folders in .\temp.
Hi, I've managed to use and run this version and I got no error, however I am not sure if I configured it right.
My logs shows no data:

Code: Select all

Wed 12/11/2019 18:29:51.94 - START 
HAM: 
SPAM: 
Wed 12/11/2019 18:29:51.95 - STOP 
Wed 12/11/2019 18:43:48.84 - START 
HAM: 
SPAM: 
Wed 12/11/2019 18:46:03.29 - STOP 
Wed 12/11/2019 18:46:34.55 - START 
HAM: 
SPAM: 
Wed 12/11/2019 18:47:41.52 - STOP 
Wed 12/11/2019 18:52:41.20 - START 
HAM: 
SPAM: 
Wed 12/11/2019 18:52:41.20 - STOP 
Wed 12/11/2019 18:55:55.38 - START 
HAM: 
SPAM: 
Wed 12/11/2019 18:55:55.38 - STOP 
Also, when I was trying to run sa-learn.vbs thru windows scheduler, I got this pop-up:
Image

I'm worried that if I set the time like 04:00am this doesn't automate because I need to click the 'ok' button to proceed. Is there any script where to automatically run the sa-learn.vbs?

My 'temp' folder has a data but the 'bayes_db' folder has no data.

And, do I need to enabled any of these inside my local.cf or just leave it as it is?

Code: Select all

#   Use Bayesian classifier (default: 1)
#
# use_bayes 1


#   Bayesian classifier auto-learning (default: 1)
#
# bayes_auto_learn 1

#   Set headers which may provide inappropriate cues to the Bayesian
#   classifier
#
# bayes_ignore_header X-Bogosity
# bayes_ignore_header X-Spam-Flag
# bayes_ignore_header X-Spam-Status

palinka
Senior user
Senior user
Posts: 1303
Joined: 2017-09-12 17:57

Re: SpamAssassin Bootcamp (sa-learn) train BAYES

Post by palinka » 2019-12-11 19:18

Re the popup:

Code: Select all

Const Verbose         = 0
You have it set to 1, I believe.

palinka
Senior user
Senior user
Posts: 1303
Joined: 2017-09-12 17:57

Re: SpamAssassin Bootcamp (sa-learn) train BAYES

Post by palinka » 2019-12-11 19:25

ashtec014 wrote:
2019-12-11 18:24
And, do I need to enabled any of these inside my local.cf or just leave it as it is?

Code: Select all

#   Use Bayesian classifier (default: 1)
#
# use_bayes 1


#   Bayesian classifier auto-learning (default: 1)
#
# bayes_auto_learn 1

#   Set headers which may provide inappropriate cues to the Bayesian
#   classifier
#
# bayes_ignore_header X-Bogosity
# bayes_ignore_header X-Spam-Flag
# bayes_ignore_header X-Spam-Status
Try this instead. Make sure to change bayes_path to the correct folder. SA windows user account needs access permission to the folder, so don't put it somewhere that SA cannot be read/write. Restart required.

Code: Select all

use_bayes 1
bayes_path X:\sa-learn\.spamassassin\bayes
bayes_auto_learn 0
# bayes_ignore_header X-Bogosity
bayes_ignore_header X-Spam-Flag
bayes_ignore_header X-Spam-Status
You DO NOT want to use autolearn with this script. I'm not even sure if it will work at all. But autolearn is the EXACT OPPOSITE of bayes training.
Last edited by palinka on 2019-12-11 19:34, edited 1 time in total.

palinka
Senior user
Senior user
Posts: 1303
Joined: 2017-09-12 17:57

Re: SpamAssassin Bootcamp (sa-learn) train BAYES

Post by palinka » 2019-12-11 19:32

Also....

https://spamassassin.apache.org/full/3. ... _Conf.html
bayes_min_ham_num (Default: 200)
bayes_min_spam_num (Default: 200)
To be accurate, the Bayes system does not activate until a certain number of ham (non-spam) and spam have been learned. The default is 200 of each ham and spam, but you can tune these up or down with these two settings.
You can change the minimum if you need to, but I think it will work better with the default minimum of 200. I trust these guys to know what's best. :mrgreen:

ashtec014
Normal user
Normal user
Posts: 41
Joined: 2019-09-05 11:56

Re: SpamAssassin Bootcamp (sa-learn) train BAYES

Post by ashtec014 » 2019-12-12 09:16

palinka wrote:
2019-12-11 19:18
Re the popup:

Code: Select all

Const Verbose         = 0
You have it set to 1, I believe.
Hi Palinka, I changed this to

Code: Select all

Const Verbose         = 1
but still I am getting the pop after running the task scheduler.

ashtec014
Normal user
Normal user
Posts: 41
Joined: 2019-09-05 11:56

Re: SpamAssassin Bootcamp (sa-learn) train BAYES

Post by ashtec014 » 2019-12-12 09:18

palinka wrote:
2019-12-11 19:32
Also....

https://spamassassin.apache.org/full/3. ... _Conf.html
bayes_min_ham_num (Default: 200)
bayes_min_spam_num (Default: 200)
To be accurate, the Bayes system does not activate until a certain number of ham (non-spam) and spam have been learned. The default is 200 of each ham and spam, but you can tune these up or down with these two settings.
You can change the minimum if you need to, but I think it will work better with the default minimum of 200. I trust these guys to know what's best. :mrgreen:
Regarding this one:
bayes_min_ham_num (Default: 200)
bayes_min_spam_num (Default: 200)
Do I need to add this as well to local.cf or where do I find this?

palinka
Senior user
Senior user
Posts: 1303
Joined: 2017-09-12 17:57

Re: SpamAssassin Bootcamp (sa-learn) train BAYES

Post by palinka » 2019-12-12 13:47

ashtec014 wrote:
2019-12-12 09:16
palinka wrote:
2019-12-11 19:18
Re the popup:

Code: Select all

Const Verbose         = 0
You have it set to 1, I believe.
Hi Palinka, I changed this to

Code: Select all

Const Verbose         = 1
but still I am getting the pop after running the task scheduler.
Should be 0 for no popup.

palinka
Senior user
Senior user
Posts: 1303
Joined: 2017-09-12 17:57

Re: SpamAssassin Bootcamp (sa-learn) train BAYES

Post by palinka » 2019-12-12 13:50

ashtec014 wrote:
2019-12-12 09:18
palinka wrote:
2019-12-11 19:32
Also....

https://spamassassin.apache.org/full/3. ... _Conf.html
bayes_min_ham_num (Default: 200)
bayes_min_spam_num (Default: 200)
To be accurate, the Bayes system does not activate until a certain number of ham (non-spam) and spam have been learned. The default is 200 of each ham and spam, but you can tune these up or down with these two settings.
You can change the minimum if you need to, but I think it will work better with the default minimum of 200. I trust these guys to know what's best. :mrgreen:
Regarding this one:
bayes_min_ham_num (Default: 200)
bayes_min_spam_num (Default: 200)
Do I need to add this as well to local.cf or where do I find this?
You don't need to add it. But you will need 200 spams and 200 hams for bayes to work. If you don't have that many and can't wait, you can add those lines to local.cf and lower the numbers. But I recommend waiting for 200.

ashtec014
Normal user
Normal user
Posts: 41
Joined: 2019-09-05 11:56

Re: SpamAssassin Bootcamp (sa-learn) train BAYES

Post by ashtec014 » 2019-12-12 14:43

palinka wrote:
2019-12-12 13:47
ashtec014 wrote:
2019-12-12 09:16
palinka wrote:
2019-12-11 19:18
Re the popup:

Code: Select all

Const Verbose         = 0
You have it set to 1, I believe.
Hi Palinka, I changed this to

Code: Select all

Const Verbose         = 1
but still I am getting the pop after running the task scheduler.
Should be 0 for no popup.
Pop-up is still exist after reverting it back to 0.

ashtec014
Normal user
Normal user
Posts: 41
Joined: 2019-09-05 11:56

Re: SpamAssassin Bootcamp (sa-learn) train BAYES

Post by ashtec014 » 2019-12-12 14:45

palinka wrote:
2019-12-12 13:50
ashtec014 wrote:
2019-12-12 09:18
palinka wrote:
2019-12-11 19:32
Also....

https://spamassassin.apache.org/full/3. ... _Conf.html



You can change the minimum if you need to, but I think it will work better with the default minimum of 200. I trust these guys to know what's best. :mrgreen:
Regarding this one:
bayes_min_ham_num (Default: 200)
bayes_min_spam_num (Default: 200)
Do I need to add this as well to local.cf or where do I find this?
You don't need to add it. But you will need 200 spams and 200 hams for bayes to work. If you don't have that many and can't wait, you can add those lines to local.cf and lower the numbers. But I recommend waiting for 200.
Thank you this is noted. So far spams are not yet 200 but hams I'm sure its already more than 200 but I'm gonna wait as recommended.

tunis
Normal user
Normal user
Posts: 228
Joined: 2015-01-05 20:22
Location: Sweden

Re: SpamAssassin Bootcamp (sa-learn) train BAYES

Post by tunis » 2019-12-12 16:06

ashtec014 wrote:
2019-12-12 14:43
palinka wrote:
2019-12-12 13:47
ashtec014 wrote:
2019-12-12 09:16


Hi Palinka, I changed this to

Code: Select all

Const Verbose         = 1
but still I am getting the pop after running the task scheduler.
Should be 0 for no popup.
Pop-up is still exist after reverting it back to 0.
Are call it with cscript?

If you use wscript you get popups.
HMS 5.6.8 B2437.17 on Windows Server 2019 Core VM.
HMS 5.6.8 B2451.21 on Windows Server 2016 Core VM.
HMS 5.6.7 B2425.16 on Windows Server 2012 R2 Core VM.

ashtec014
Normal user
Normal user
Posts: 41
Joined: 2019-09-05 11:56

Re: SpamAssassin Bootcamp (sa-learn) train BAYES

Post by ashtec014 » 2019-12-12 16:17

tunis wrote:
2019-12-12 16:06
ashtec014 wrote:
2019-12-12 14:43
palinka wrote:
2019-12-12 13:47


Should be 0 for no popup.
Pop-up is still exist after reverting it back to 0.
Are call it with cscript?

If you use wscript you get popups.
Unfortunately no script. I just normally called it thru windows scheduler. I have no idea how to use cscript. Can you please help me how? Thank you

palinka
Senior user
Senior user
Posts: 1303
Joined: 2017-09-12 17:57

Re: SpamAssassin Bootcamp (sa-learn) train BAYES

Post by palinka » 2019-12-12 16:46

Here's what I have. I only changed the password and domain names.

Code: Select all

Option Explicit
'
' Version 0.6.1 28/07-2018, Soren Rathje - Curley brackets still acting up, reverting to copying mails.
' Version 0.6.0 26/07-2018, Soren Rathje - Experimental support for both sa-learn AND spamc -L.
' Version 0.5.a 26/07-2018, Soren Rathje - Experimental rewrite of filelist to fix curly brace problem in sa-learn.
' Version 0.5.0 25/07-2018, Soren Rathje - Multiple domains, skipping non-existing folders plus reworked code.
' Version 0.4.3 30/05-2018, Soren Rathje - Introduced two special folders; non-delivered SPAM and False Positives.
' Version 0.4.2 28/05-2016, Soren Rathje - Compatibility issues (curly brackets bug in sa-learn).
' Version 0.4.1 25/11-2014, Soren Rathje - Compatibility issues.
' Version 0.4.0 27/10-2014, Soren Rathje - Changed error logging.
' Version 0.3.0 11/10-2014, Soren Rathje - Bugfixing & Log error to Eventlog if IMAPFolder is missing
' Version 0.2.0 30/08-2014, Soren Rathje - Selection changed to DAYS.
' Version 0.1.0 09-08-2014, Soren Rathje - Initial version.
'
' Configuration parameters
'
'     Administrative and automation accounts can be excluded from processing
'     by defining them in "ExcludeList"
'
Const Administrator   = "Administrator"
Const Secret          = "supersecretpassword"

Const ExcludeList     = "user@mydomain.tld, user@anotherdomain.tld"
Const DomainList      = "mydomain.tld, anotherdomain.tld, thirddomain.tld"
Const SPAMFolders     = "SPAM, Junk, Junk E-mail"
Const HAMFolders      = "INBOX, HAM"

Const Batch           = 1                              ' 0 = spamc, 1 = sa-learn
Const SALearn         = "X:\sa-learn\sa-learn.cmd" ' Intermediate commandfile
Const TempDir         = "X:\sa-learn\temp\"        ' Need permission for create, read & write
Const LogDir          = "C:\SpamAssassin\logs\"        ' Need permission for create, read & write
Const BayesDir        = "X:\sa-learn\.spamassassin\"    ' Need permission for create, read & write
Const RetainDays      = 7
Const Verbose         = 0

Sub BuildList(a, mExcludes, b, mDays, mTemp, mType)
   Dim i, j, k, l, strFileName
   Dim oFile, oDomain, oAccount, oMessage, oMessages
   Dim mDomain, mDomains, mFolder, mFolders
   
   If Batch Then Set oFile = oFSO.CreateTextFile(mTemp & mType & ".cmd",True)
   mDomains = Split(a, ",")
   mFolders = Split(b, ",")
   If Verbose Then WScript.Echo "Type: " & mType
   For Each mDomain In mDomains
      mDomain = Trim(mDomain)
      Set oDomain = oApp.Domains.ItemByName(mDomain)
      If Verbose Then WScript.Echo "     Domain: " & oDomain.Name
      For i = 0 To oDomain.Accounts.Count - 1
         Set oAccount = oDomain.Accounts.Item(i)
         If InStr(mExcludes, oAccount.Address) = 0 And oAccount.Active Then
            If Verbose Then WScript.Echo "          Account: " & oAccount.Address
            For Each mFolder In mFolders
               mFolder = Trim(mFolder)
               On Error Resume Next
               If oAccount.IMAPFolders.ItemByName(mFolder) Is Nothing Then
                  On Error Goto 0
               Else
                  On Error Goto 0
                  If Verbose Then WScript.Echo "               Folder: " & mFolder
                  Set oMessages  = oAccount.IMAPFolders.ItemByName(mFolder).Messages
                  If Not IsNull(oMessages) Then
                     For j = 0 To oMessages.Count - 1
                        Set oMessage = oMessages.Item(j)
                        If oMessage.InternalDate > CDate(Now - mDays) Then
                           If Batch Then
                              oFile.Write "COPY " & Chr(34) & oMessage.FileName & Chr(34) & " " & mTemp & mType & "\" & CLng(oMessage.ID) & ".eml /Y" & vbCrLf
                           Else
                              oCMD.Run "cmd.exe /C spamc.exe -d " & SAHost & " -p " & SAPort & " -L " & mType & " < " & Chr(34) & oMessage.FileName & Chr(34), Verbose, True
                           End If
                        End If
                     Next
                  End If
               End If
            Next
         End If
      Next
   Next
   If Batch Then oFile.Close
End Sub

Dim oFSO : Set oFSO = CreateObject("Scripting.FileSystemObject")
Dim oApp : Set oApp = CreateObject("hMailServer.Application")
Dim oCMD : Set oCMD = CreateObject("WScript.Shell")
Call oApp.Authenticate(Administrator, Secret)
Dim SAHost : SAHost = oApp.Settings.AntiSpam.SpamAssassinHost
Dim SAPort : SAPort = oApp.Settings.AntiSpam.SpamAssassinPort

'
' Find HAM messages
'
If Verbose Then WScript.Echo "Processing HAM mails"
Call BuildList(DomainList, ExcludeList, HAMFolders, RetainDays, TempDir, "ham")

'
' Find SPAM messages
'
If Verbose Then WScript.Echo "Processing SPAM mails"
Call BuildList(DomainList, ExcludeList, SPAMFolders, RetainDays, TempDir, "spam")

'
' Execute file copy and sa-learn.exe - sequentially - no StdOut.
'
If Batch Then
   ' On Error Resume Next
   
   If Verbose Then WScript.Echo "Copying SPAM mails"
   oCMD.Run "cmd.exe /C " & TempDir & "spam.cmd", Verbose, True
   If Err.Number Then
      EventLog.Write ("Exception   : sa-learn.vbs -> oCMD.Run SPAMCopy, 0, true")
      EventLog.Write ("Error       : " & Err.Number)
      EventLog.Write ("Error (hex) : 0x" & Hex(Err.Number))
      EventLog.Write ("Source      : " & Err.Source)
      EventLog.Write ("Description : " & Err.Description)
      Err.Clear
   End If
   
   If Verbose Then WScript.Echo "Copying HAM mails"
   oCMD.Run "cmd.exe /C " & TempDir & "ham.cmd", Verbose, True
   If Err.Number Then
      EventLog.Write ("Exception   : sa-learn.vbs -> oCMD.Run HAMCopy, 0, true")
      EventLog.Write ("Error       : " & Err.Number)
      EventLog.Write ("Error (hex) : 0x" & Hex(Err.Number))
      EventLog.Write ("Source      : " & Err.Source)
      EventLog.Write ("Description : " & Err.Description)
      Err.Clear
   End If
   
   If Verbose Then WScript.Echo "Starting the learning process ... "
   oCMD.Run "cmd.exe /C " & SALearn, Verbose, True
   If Err.Number Then
      EventLog.Write ("Exception   : sa-learn.vbs -> oCMD.Run SALearn, 0, true")
      EventLog.Write ("Error       : " & Err.Number)
      EventLog.Write ("Error (hex) : 0x" & Hex(Err.Number))
      EventLog.Write ("Source      : " & Err.Source)
      EventLog.Write ("Description : " & Err.Description)
      Err.Clear
   End If
   
   ' On Error Goto 0
   
End If

As you can see, by setting "Const Verbose = 0" it will bypass lines like:

Code: Select all

   If Verbose Then WScript.Echo "Starting the learning process ... "
If that's not happening for you, then you can delete all the verbose if statements. But you probably have some silly mistake like your scheduled task is using a different file than the one you're editing, or you didn't save it, or something dumb like that.

tunis
Normal user
Normal user
Posts: 228
Joined: 2015-01-05 20:22
Location: Sweden

Re: SpamAssassin Bootcamp (sa-learn) train BAYES

Post by tunis » 2019-12-12 17:02

ashtec014 wrote:
2019-12-12 16:17
tunis wrote:
2019-12-12 16:06
ashtec014 wrote:
2019-12-12 14:43


Pop-up is still exist after reverting it back to 0.
Are call it with cscript?

If you use wscript you get popups.
Unfortunately no script. I just normally called it thru windows scheduler. I have no idea how to use cscript. Can you please help me how? Thank you
You call it like this in windows scheduler: "cscript.exe" C:\path\sa-learn.vbs

Code: Select all

schtasks /create /ru "SYSTEM" /tn "SA learn" /tr "\"cscript.exe\" C:\path\sa-learn.vbs" /sc DAILY /mo 1
HMS 5.6.8 B2437.17 on Windows Server 2019 Core VM.
HMS 5.6.8 B2451.21 on Windows Server 2016 Core VM.
HMS 5.6.7 B2425.16 on Windows Server 2012 R2 Core VM.

ashtec014
Normal user
Normal user
Posts: 41
Joined: 2019-09-05 11:56

Re: SpamAssassin Bootcamp (sa-learn) train BAYES

Post by ashtec014 » 2019-12-14 09:03

tunis wrote:
2019-12-12 17:02
You call it like this in windows scheduler: "cscript.exe" C:\path\sa-learn.vbs

Code: Select all

schtasks /create /ru "SYSTEM" /tn "SA learn" /tr "\"cscript.exe\" C:\path\sa-learn.vbs" /sc DAILY /mo 1
This one works for me. I ran it via command prompt then attempted to run using windows scheduler and got the logs. Thanks so much guys! I really appreciate your help.

Post Reply