Page 2 of 2

Re: SpamAssassin Custom Rules

Posted: 2016-03-03 23:57
by jimimaseye
SorenR wrote:
RvdH wrote:Version 3.4.1.36: (03 March 2016)
Wohoo - That was about bloody time :mrgreen:

"Support for Windows XP and Server 2003 has been dropped." Bummer :evil:
Soren, you should be aware of this:

https://fossies.org/diffs/Mail-SpamAssa ... -diff.html
  • - When Bayes classification is in use and messages are 'learned' as spam
    or ham and stored in a database, the Bayes plugin generates internal
    message IDs of learned messages and stores them in a 'seen' database to
    avoid re-learning duplicates and accidental un-learning messages that
    were not previously learned. With changes in bug 5185, the calculation
    of message IDs in a bayes 'seen' database has changed, so new code can
    no longer associate new messages with those learned before the change.
  • - Note that this change does not affect recognition of old tokens and the
    classification algorithm, only duplicate detection and unlearning of old
    messages is affected.
  • - Because of this change, if you use Bayes and you are upgrading from a
    version prior to 3.4.0, you may consider wiping your Bayes database
    and starting fresh.
But it does say versions PRIOR to 3.4.0

Re: SpamAssassin Custom Rules

Posted: 2016-03-04 00:13
by SorenR
jimimaseye wrote:
SorenR wrote:
RvdH wrote:Version 3.4.1.36: (03 March 2016)
Wohoo - That was about bloody time :mrgreen:

"Support for Windows XP and Server 2003 has been dropped." Bummer :evil:
Soren, you should be aware of this: ............
Erm... Well... It's going to cost me an upgrade to my laptop (Dell D610) to some i7 quad core monster and some kind of 64 bit OS upgrade to my server. Not sure I will go for server version, I may settle for Win10 Pro... All it does is hMailServer... All my other stuff run on Linux systems about 1/10'th the size :mrgreen:

Re: SpamAssassin Custom Rules

Posted: 2016-03-04 01:20
by mattg
f you are 'just' running hMailserver, think about the HyperV server Core OS.

No GUI, but the hmailserver GUI shows and works fine.
Free download.

Install MySQL, dotNET components, install hMailserver and good to go.
Can install Appache too if needed. IIS doesn't work.

This OS can also run HyperV VMs (that is what it is designed to do)

Re: SpamAssassin Custom Rules

Posted: 2017-08-04 21:25
by RvdH
jimimaseye wrote:Sorry to come back to this.... (I am doing so because today for a week or so I am turning on Direct deliveries in instead of using External Download)....

Although we acknowledge that the SPF test is failing due to the lack of RETURN-PATH header at time of being passed to SA, and that adding "envelope_sender_header From " in to local.cf gives it something to check and therefore report on (albeit not too guaranteed relialke due to blank or spoofing opportunities), but really.... is it worth it?! Ive just noticed that despite it all,the SPF_PASS check that spamassassin does still only scores it as ZERO for a positive pass.

Code: Select all

	* -0.0 SPF_PASS SPF: sender matches SPF record
Its true, that a fail might give a positive score (increasing the scoring towards a 'spam' conclusion), and given that the from address frankly could be anything and is possible to result in a positive score under these circumstances, it simply cant be trusted to bring any genuine reliable value to the equation.

I think its best to be left out (in absence of the RETURN-PATH field)
Sorry to bring this up again, but i was reading this topic once more, the reason to have both SPF_PASS and RETURN-PATH headers was to do some custom scripting in SA, look at the example i started this topic with, mainly in UNPHISH part
RvdH wrote:

Code: Select all

# Domains:
# abnamro.nl 
# nl.abnamro.com
describe PHISH_FROM_ABNAMRO        Trigger on phishing mails
header   PHISH_FROM_ABNAMRO        From:addr =~ /\@(abnamro\.nl|nl\.abnamro\.com)$/i
score    PHISH_FROM_ABNAMRO        2.0

describe UNPHISH_FROM_ABNAMRO      Untrigger on valid mails
header   __UNPHISH_FROM_ABNAMRO_A  Return-Path:addr =~ /\@(abnamro\.nl|nl\.abnamro\.com)$/i
meta     UNPHISH_FROM_ABNAMRO      ( __UNPHISH_FROM_ABNAMRO_A && SPF_PASS )
score    UNPHISH_FROM_ABNAMRO      -2.0
Anyway, i do this different now, and most Dutch banks finally use DKIM, eg:

is score much higher on phishing domains, eg:

Code: Select all

describe PHISH_FROM_ABNAMRO        Trigger on phishing mails
header   PHISH_FROM_ABNAMRO        From:addr =~ /\@(.+)?(abnamro\.nl|nl\.abnamro\.com)$/i
score    PHISH_FROM_ABNAMRO        10.0
and then: (score: -100)

whitelist_auth *@abnamro.nl
whitelist_auth *@mail.abnamro.nl
whitelist_auth *@nl.abnamro.com

Code: Select all

       whitelist_auth add@ress.com
            Used to specify addresses which send mail that is often tagged
            (incorrectly) as spam. This is different from "whitelist_from"
            and "whitelist_from_rcvd" in that it first verifies that the
            message was sent by an authorized sender for the address, before
            whitelisting.

            Authorization is performed using one of the installed
            sender-authorization schemes: SPF (using
            "Mail::SpamAssassin::Plugins::SPF"), Domain Keys (using
            "Mail::SpamAssassin::Plugins::DomainKeys"), or DKIM (using
            "Mail::SpamAssassin::Plugins::DKIM"). Note that those plugins
            must be active, and working, for this to operate.

            Using "whitelist_auth" is roughly equivalent to specifying
            duplicate "whitelist_from_spf", "whitelist_from_dk", and
            "whitelist_from_dkim" lines for each of the addresses specified.

Re: SpamAssassin Custom Rules

Posted: 2017-08-05 13:04
by RvdH
I managed to alter the code to Include the Return-Path header before sending it to SA...
Only now it has 2 Return-Path headers :oops: :lol:

Code: Select all

Return-Path: sender@domain.com
Delivered-To: recipient@domain.com
Return-Path: sender@domain.com
X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on WIN-2016-STD
X-Spam-Level: ...
Mmm, think i have to find a way to remove the added Return-Path header after SA finishes....the quest continues :wink:

Re: SpamAssassin Custom Rules

Posted: 2017-08-05 15:08
by RvdH
SpamTestSpamAssassin.cpp

Code: Select all

// Copyright (c) 2010 Martin Knafve / hMailServer.com.  
// http://www.hmailserver.com

#include "StdAfx.h"

#include "SpamTestSpamAssassin.h"

#include "SpamTestData.h"
#include "SpamTestResult.h"

#include "AntiSpamConfiguration.h"

#include "SpamAssassin/SpamAssassinClient.h"

#include "../TCPIP/IOService.h"
#include "../TCPIP/TCPConnection.h"
#include "../TCPIP/SslContextInitializer.h"
#include "../TCPIP/DNSResolver.h"

#include "../BO/MessageData.h"
#include "../BO/Message.h"
#include "../Util/TraceHeaderWriter.h"
#include "../Util/event.h"
#include "../Persistence/PersistentMessage.h"


#ifdef _DEBUG
#define DEBUG_NEW new(_NORMAL_BLOCK, __FILE__, __LINE__)
#define new DEBUG_NEW
#endif

namespace HM
{
   String 
   SpamTestSpamAssassin::GetName() const
   {
      return "SpamTestSpamAssassin";
   }

   bool 
   SpamTestSpamAssassin::GetIsEnabled()
   {
      AntiSpamConfiguration &config = Configuration::Instance()->GetAntiSpamConfiguration();
      if (config.GetSpamAssassinEnabled())
         return true;
      else
         return false;
   }

   std::set<std::shared_ptr<SpamTestResult> >
   SpamTestSpamAssassin::RunTest(std::shared_ptr<SpamTestData> pTestData)
   {
      std::set<std::shared_ptr<SpamTestResult> > setSpamTestResults;
      
      AntiSpamConfiguration& config = Configuration::Instance()->GetAntiSpamConfiguration();

      std::shared_ptr<Message> pMessage = pTestData->GetMessageData()->GetMessage();
      const String sFilename = PersistentMessage::GetFileName(pMessage);

	  // Add Return-Path header.
	  std::vector<std::pair<AnsiString, AnsiString> > fieldsToWrite;
	  fieldsToWrite.push_back(std::make_pair("Return-Path", pMessage->GetFromAddress()));
	  TraceHeaderWriter writer;
	  writer.Write(sFilename, pMessage, fieldsToWrite);	
      
      std::shared_ptr<IOService> pIOService = Application::Instance()->GetIOService();

      bool testCompleted;

      std::shared_ptr<Event> disconnectEvent = std::shared_ptr<Event>(new Event());
      std::shared_ptr<SpamAssassinClient> pSAClient = std::shared_ptr<SpamAssassinClient>(new SpamAssassinClient(sFilename, pIOService->GetIOService(), pIOService->GetClientContext(), disconnectEvent, testCompleted));
      
      String sHost = config.GetSpamAssassinHost();
      int iPort = config.GetSpamAssassinPort();

      
      DNSResolver resolver;

      std::vector<String> ip_addresses;
      resolver.GetARecords(sHost, ip_addresses);

      String ip_address;
      if (ip_addresses.size())
      {
         ip_address = *(ip_addresses.begin());
      }
      else
      {
         String message = "The IP address for SpamAssassin could not be resolved. Aborting tests.";
         ErrorManager::Instance()->ReportError(ErrorManager::High, 5507, "SpamAssassinTestConnect::TestConnect", message);
         return setSpamTestResults;  
      }

      // Here we handle of the ownership to the TCPIP-connection layer.
      if (pSAClient->Connect(ip_address, iPort, IPAddress()))
      {
         // Make sure we keep no references to the TCP connection so that it
         // can be terminated whenever. We're longer own the connection.
         pSAClient.reset();

         disconnectEvent->Wait();
      }
      
      if (!testCompleted)
      {
         ErrorManager::Instance()->ReportError(ErrorManager::High, 5508, "SpamAssassinTestConnect::TestConnect", 
            "The SpamAssassin tests did not complete. Please confirm that the configuration (host name and port) is valid and that SpamAssassin is running.");

         return setSpamTestResults;  
      }

      // Check if the message is tagged as spam.
      std::shared_ptr<MessageData> pMessageData = pTestData->GetMessageData();
	  pMessageData->RefreshFromMessage();
	  // Remove Return-Path header.
	  pMessageData->DeleteField("Return-Path");
	  pMessageData->Write(sFilename);

      bool bIsSpam = false;
      AnsiString sSpamStatus = pMessageData->GetFieldValue("X-Spam-Status");
      if (sSpamStatus.Mid(0, 3).ToUpper() == "YES")
         bIsSpam = true;

      if (bIsSpam)
      {
         int iScore = 0;
         if (config.GetSpamAssassinMergeScore())
            iScore = ParseSpamAssassinScore_(sSpamStatus);
         else
            iScore = config.GetSpamAssassinScore();

         String sMessage = "Tagged as Spam by SpamAssassin";
         std::shared_ptr<SpamTestResult> pResult = std::shared_ptr<SpamTestResult>(new SpamTestResult(GetName(), SpamTestResult::Fail, iScore, sMessage));
         setSpamTestResults.insert(pResult);   
      }
      
      return setSpamTestResults;
   }

   int 
   SpamTestSpamAssassin::ParseSpamAssassinScore_(const AnsiString &sHeader)
   {
      int iStartPos = sHeader.FindNoCase("score=");
      if (iStartPos < 0)
         return 0;

      iStartPos += 6;

      int iScoreEnd = sHeader.Find(".", iStartPos);
      if (iScoreEnd < 0)
         return 0;

      int iScoreLength = iScoreEnd - iStartPos;

      if (iScoreLength <= 0)
         return 0;

      AnsiString sScore = sHeader.Mid(iStartPos, iScoreLength);

      int iRetVal = atoi(sScore);
      return iRetVal;

   }




}

Re: SpamAssassin Custom Rules

Posted: 2017-08-05 19:20
by RvdH
Little fix, only Add Return-Path if none exists, External Account downloaded message might (should!) already have one

in above SpamTestSpamAssassin.cpp change this

Code: Select all

	  // Add Return-Path header if none exist (ExternalAccount download?)
	  if (pTestData->GetMessageData()->GetReturnPath().IsEmpty())
	  {
		  std::vector<std::pair<AnsiString, AnsiString> > fieldsToWrite;
		  fieldsToWrite.push_back(std::make_pair("Return-Path", pMessage->GetFromAddress()));
		  TraceHeaderWriter writer;
		  writer.Write(sFilename, pMessage, fieldsToWrite);
	  }

Re: SpamAssassin Custom Rules

Posted: 2017-08-05 20:26
by RvdH
Probably more important fix :oops: :lol:

Code: Select all

	  // Add Return-Path header if none exist (ExternalAccount download?)
	  if (pTestData->GetMessageData()->GetReturnPath().IsEmpty())
	  {
		  std::vector<std::pair<AnsiString, AnsiString> > fieldsToWrite;
		  fieldsToWrite.push_back(std::make_pair("Return-Path", pTestData->GetEnvelopeFrom()));
		  TraceHeaderWriter writer;
		  writer.Write(sFilename, pMessage, fieldsToWrite);
	  }