Sub OnHELO(oClient) progress?

Use this forum if you want to discuss a problem or ask a question related to a hMailServer beta release.
User avatar
SorenR
Senior user
Senior user
Posts: 2327
Joined: 2006-08-21 15:38
Location: Denmark

Sub OnHELO(oClient) progress?

Postby SorenR » 2016-08-10 18:11

Well... Not sure how this is progressing so...

Moved my personal changes forward from 5.4.2-B1964 to 5.6.5-B2367. Those of you with programming genes know how to work this file. Not a Git-Geek - did SVN back in the day, worked mostly on 'nix, thus the "diff" ;-)

Pitfalls (YES, there is...) If you assign ANY value to "Return.Value" in "Sub OnHELO(oClient)" the server WILL crash. Here, I've said it, be warned!

Code: Select all

diff -bur Z:\hMailServer-5.6.5\B2367/ScriptServer.cpp Z:\hMailServer-5.6.5\B2367.1/ScriptServer.cpp
--- Z:\hMailServer-5.6.5\B2367/ScriptServer.cpp   2016-05-24 13:12:26.000000000 +0200
+++ Z:\hMailServer-5.6.5\B2367.1/ScriptServer.cpp   2016-08-10 13:38:00.999999900 +0200
@@ -29,7 +29,8 @@
       has_on_error_(false),
       has_on_delivery_failed_(false),
       has_on_external_account_download_(false),
-      has_on_smtpdata_(false)
+      has_on_smtpdata_(false),
+      has_on_helo_(false)
    {
       
    }
@@ -97,6 +98,7 @@
          has_on_delivery_failed_ = DoesFunctionExist_("OnDeliveryFailed");
          has_on_external_account_download_ = DoesFunctionExist_("OnExternalAccountDownload");
          has_on_smtpdata_ = DoesFunctionExist_("OnSMTPData");
+         has_on_helo_ = DoesFunctionExist_("OnHELO");
 
       }
       catch (...)
@@ -251,6 +253,12 @@
             return;
          break;
 
+      case EventOnHELO:
+            event_name_ = _T("OnHELO");
+            if (!has_on_helo_)
+            return;
+         break;
+
       case EventCustom:
          break;
       default:
diff -bur Z:\hMailServer-5.6.5\B2367/ScriptServer.h Z:\hMailServer-5.6.5\B2367.1/ScriptServer.h
--- Z:\hMailServer-5.6.5\B2367/ScriptServer.h   2016-05-24 13:12:26.000000000 +0200
+++ Z:\hMailServer-5.6.5\B2367.1/ScriptServer.h   2016-08-10 13:02:02.999999900 +0200
@@ -28,6 +28,7 @@
         
          EventOnExternalAccountDownload = 1011,
          EventOnSMTPData = 1012,
+         EventOnHELO = 1013,
       };
 
       ScriptServer(void);
@@ -64,6 +65,7 @@
       bool has_on_delivery_failed_;
       bool has_on_external_account_download_;
       bool has_on_smtpdata_;
+      bool has_on_helo_;
 
       String script_contents_;
       String script_extension_;
diff -bur Z:\hMailServer-5.6.5\B2367/SMTPConnection.cpp Z:\hMailServer-5.6.5\B2367.1/SMTPConnection.cpp
--- Z:\hMailServer-5.6.5\B2367/SMTPConnection.cpp   2016-05-24 13:12:26.000000000 +0200
+++ Z:\hMailServer-5.6.5\B2367.1/SMTPConnection.cpp   2016-08-10 13:30:54.999999900 +0200
@@ -1513,6 +1513,51 @@
          return;
       }
 
+      //
+      // Event OnHELO
+      //
+      if (Configuration::Instance()->GetUseScriptServer())
+      {
+         std::shared_ptr<ScriptObjectContainer> pContainer = std::shared_ptr<ScriptObjectContainer>(new ScriptObjectContainer);
+         std::shared_ptr<Result> pResult = std::shared_ptr<Result>(new Result);
+         std::shared_ptr<ClientInfo> pClientInfo = std::shared_ptr<ClientInfo>(new ClientInfo);
+
+         pClientInfo->SetIPAddress(GetIPAddressString());
+         pClientInfo->SetPort(GetLocalEndpointPort());
+         pClientInfo->SetHELO(helo_host_);
+
+         pContainer->AddObject("HMAILSERVER_CLIENT", pClientInfo, ScriptObject::OTClient);
+         pContainer->AddObject("Result", pResult, ScriptObject::OTResult);
+
+         String sEventCaller = "OnHELO(HMAILSERVER_CLIENT)";
+         ScriptServer::Instance()->FireEvent(ScriptServer::EventOnHELO, sEventCaller, pContainer);
+
+         switch (pResult->GetValue())
+         {
+            case 1:
+            {
+               String sErrorMessage = "554 Rejected";
+               EnqueueWrite_(sErrorMessage);
+               LogAwstatsMessageRejected_();
+               return;
+            }
+            case 2:
+            {
+               String sErrorMessage = "554 " + pResult->GetMessage();
+               EnqueueWrite_(sErrorMessage);
+               LogAwstatsMessageRejected_();
+               return;
+            }
+            case 3:
+            {
+               String sErrorMessage = "453 " + pResult->GetMessage();
+               EnqueueWrite_(sErrorMessage);
+               LogAwstatsMessageRejected_();
+               return;
+            }
+         }
+      }
+
       SendEHLOKeywords_();
 
       if (current_state_ == INITIAL)
@@ -1531,6 +1576,51 @@
          return;
       }
 
+      //
+      // Event OnHELO
+      //
+      if (Configuration::Instance()->GetUseScriptServer())
+      {
+         std::shared_ptr<ScriptObjectContainer> pContainer = std::shared_ptr<ScriptObjectContainer>(new ScriptObjectContainer);
+         std::shared_ptr<Result> pResult = std::shared_ptr<Result>(new Result);
+         std::shared_ptr<ClientInfo> pClientInfo = std::shared_ptr<ClientInfo>(new ClientInfo);
+
+         pClientInfo->SetIPAddress(GetIPAddressString());
+         pClientInfo->SetPort(GetLocalEndpointPort());
+         pClientInfo->SetHELO(helo_host_);
+
+         pContainer->AddObject("HMAILSERVER_CLIENT", pClientInfo, ScriptObject::OTClient);
+         pContainer->AddObject("Result", pResult, ScriptObject::OTResult);
+
+         String sEventCaller = "OnHELO(HMAILSERVER_CLIENT)";
+         ScriptServer::Instance()->FireEvent(ScriptServer::EventOnHELO, sEventCaller, pContainer);
+
+         switch (pResult->GetValue())
+         {
+            case 1:
+            {
+               String sErrorMessage = "554 Rejected";
+               EnqueueWrite_(sErrorMessage);
+               LogAwstatsMessageRejected_();
+               return;
+            }
+            case 2:
+            {
+               String sErrorMessage = "554 " + pResult->GetMessage();
+               EnqueueWrite_(sErrorMessage);
+               LogAwstatsMessageRejected_();
+               return;
+            }
+            case 3:
+            {
+               String sErrorMessage = "453 " + pResult->GetMessage();
+               EnqueueWrite_(sErrorMessage);
+               LogAwstatsMessageRejected_();
+               return;
+            }
+         }
+      }
+
       EnqueueWrite_("250 Hello.");
 
       if (current_state_ == INITIAL)
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
RvdH
Senior user
Senior user
Posts: 458
Joined: 2008-06-27 14:42
Location: Netherlands

Re: Sub OnHELO(oClient) progress?

Postby RvdH » 2016-08-10 19:29

Thanks, now i get it...last changes should have been inside SMTPConnection.cpp and not in ScriptServer.cpp as documented here
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
SorenR
Senior user
Senior user
Posts: 2327
Joined: 2006-08-21 15:38
Location: Denmark

Re: Sub OnHELO(oClient) progress?

Postby SorenR » 2016-08-10 19:54

RvdH wrote:Thanks, now i get it...last changes should have been inside SMTPConnection.cpp and not in ScriptServer.cpp as documented here

Oops... :oops: :mrgreen:
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
SorenR
Senior user
Senior user
Posts: 2327
Joined: 2006-08-21 15:38
Location: Denmark

Re: Sub OnHELO(oClient) progress?

Postby SorenR » 2016-08-11 17:47

Hmm... There has been a development... Something works when it really should not :mrgreen:

When I initially made this for my 5.4.2 I could not use "Result.Value" and "Result.Message" as the server would crash ... :roll:

When I ported my changes to a fresh 5.6.5-B2367 I assumed (I know... Assumption Is The Mother Of All Fuckups!) it would behave the same way but I left the code in there. Well, just played a bit with it - no errors!

Code: Select all

"DEBUG"   868   "2016-08-11 17:30:02.649"   "Executing event OnHELO"
"DEBUG"   868   "2016-08-11 17:30:02.649"   "Event completed"
"SMTPD"   868   166   "2016-08-11 17:30:02.649"   "127.0.0.1"   "SENT: 554 Whooa... Whaz' up?"


Code: Select all

   Sub OnHELO(oClient)
      Result.Message = "Whooa... Whaz' up?"
      Result.Value = 2
   End Sub


If someone wants to try it out I have a fresh compiled 5.6.5-B2367.1 "hMailServer.exe" on my webserver...
http://www.lolle.org/images/hmailserver/hmailserver.rar
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
jimimaseye
Moderator
Moderator
Posts: 6402
Joined: 2011-09-08 17:48

Re: Sub OnHELO(oClient) progress?

Postby jimimaseye » 2016-08-11 21:51

Thanks soren. I'm currently away on hols but when I get back to it I will be more than happy to test/use it. Will you keep the link available for some weeks?
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
SorenR
Senior user
Senior user
Posts: 2327
Joined: 2006-08-21 15:38
Location: Denmark

Re: Sub OnHELO(oClient) progress?

Postby SorenR » 2016-08-11 23:03

jimimaseye wrote:Thanks soren. I'm currently away on hols but when I get back to it I will be more than happy to test/use it. Will you keep the link available for some weeks?

If not, you have my email :wink:
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
jimimaseye
Moderator
Moderator
Posts: 6402
Joined: 2011-09-08 17:48

Re: Sub OnHELO(oClient) progress?

Postby jimimaseye » 2016-08-12 00:01

Out of interest is it an Install package or will it be a straight manual hmailserver.exe service program swapout (to replace the official one currently installed in the program directory)?
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
SorenR
Senior user
Senior user
Posts: 2327
Joined: 2006-08-21 15:38
Location: Denmark

Re: Sub OnHELO(oClient) progress?

Postby SorenR » 2016-08-12 10:30

Manual swap.
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
RvdH
Senior user
Senior user
Posts: 458
Joined: 2008-06-27 14:42
Location: Netherlands

Re: Sub OnHELO(oClient) progress?

Postby RvdH » 2016-08-13 15:00

Works like a charm!

"TCPIP" 7048 "2016-08-13 14:56:39.506" "TCP - 213.131.38.246 connected to *.*.*.*:25."
"SMTPD" 7048 55166 "2016-08-13 14:56:39.506" "213.131.38.246" "SENT: 220 mail.domain.com ESMTP"
"SMTPD" 7748 55166 "2016-08-13 14:56:39.584" "213.131.38.246" "RECEIVED: EHLO ylmf-pc"
"SMTPD" 7748 55166 "2016-08-13 14:56:39.599" "213.131.38.246" "SENT: 554 Back off!"
"TCPIP" 7048 "2016-08-13 14:56:48.803" "TCP - 213.131.38.246 connected to *.*.*.*:25."
"SMTPD" 7048 55167 "2016-08-13 14:56:48.803" "213.131.38.246" "SENT: 220 mail.domain.com ESMTP"
"SMTPD" 6420 55167 "2016-08-13 14:56:48.897" "213.131.38.246" "RECEIVED: EHLO ylmf-pc"
"SMTPD" 6420 55167 "2016-08-13 14:56:48.913" "213.131.38.246" "SENT: 554 Back off!"


CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
SorenR
Senior user
Senior user
Posts: 2327
Joined: 2006-08-21 15:38
Location: Denmark

Re: Sub OnHELO(oClient) progress?

Postby SorenR » 2016-08-13 17:54

He he ;—)
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
RvdH
Senior user
Senior user
Posts: 458
Joined: 2008-06-27 14:42
Location: Netherlands

Re: Sub OnHELO(oClient) progress?

Postby RvdH » 2016-08-13 18:06

I have been fiddling around with your AddGreyList Sub inside OnHELO(oClient) event in EventHandlers.vbs

With below changes it will verify the ip against valid domain spf ip ranges en should block faked HELO/EHLO headers to be whitelisted

Code: Select all

   Sub OnHELO(oClient)
      If (Left(oClient.IPAddress, 10) = "192.168.0.") Then Exit Sub
      If (Left(oClient.IPAddress, 10) = "80.160.77.") Then Exit Sub
      If (oClient.Port = 25) Then Wait(20)

      Dim oRegEx
      Set oRegEx = CreateObject("VBScript.RegExp")
      oRegEx.IgnoreCase = True
      oRegEx.Global = False

      oRegEx.Pattern= "^([a-z]+[0-9]{2}\-[a-z]{2}[0-9]\-)(obe\.outbound\.protection\.outlook\.com)$|" &_
                  "^(mail\-[a-z]{2}[0-9]\-f[0-9]{1,3})(\.google\.com)$|" &_
                  "^(mail[a-z]\-[a-z]{2})(\.linkedin\.com)$|" &_
                  "^(mx\-out\.facebook\.com)$|" &_
                  "^(mail[a-z]{1}\-[a-z]{2})(\.linkedin\.com)$|" &_
                  "^(spruce\-goose\-[a-z]{2}|spring\-chicken\-[a-z]{2})(\.twitter\.com)$|" &_
                  "^([a-z]{3}[\d]{3}\-[a-z]{2,3}[\d]{1}[a-z]{1}[\d]{1,2})(\.hotmail\.com)$"
                  
      If oRegEx.Test(oClient.HELO) Then Call AddGreyList(oClient.IPAddress, oClient.HELO)

      ...

      Set oRegEx = Nothing.
   End Sub


Code: Select all

Function getDomainName(byVal strHELO)
   dim aryDomain, str2ndLevel, strTopLevel
   getDomainName = ""
   If Len(strHELO) > 0 Then     
      aryDomain = Split(strHELO,".")
      If uBound(aryDomain) >= 1 Then
         str2ndLevel = aryDomain(uBound(aryDomain)-1)
         strTopLevel = aryDomain(uBound(aryDomain))         
         getDomainName = str2ndLevel & "." & strTopLevel
      End If
   End If
End Function


Code: Select all

Sub AddGreyList(ByVal strIP, ByVal strHELO)
 
   Dim oRegEx
   Set oRegEx = CreateObject("VBScript.RegExp")
   oRegEx.IgnoreCase = True
   oRegEx.Global = False
   
   Select Case getDomainName(strHELO)
   
   case "hotmail.com"
      ' https://mail.live.com/mail/ipspace.aspx
      oRegEx.Pattern= "^65\.54\.190\.([0-9]|[1-5][0-9]|6[0-3])$|" &_
                  "^65\.54\.190\.(6[4-9]|[7-9][0-9]|1([0-1][0-9]|2[0-7]))$|" &_
                  "^65\.54\.190\.(1(2[8-9]|[3-8][0-9]|9[0-1]))$|" &_
                  "^65\.54\.190\.(1(9[2-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^65\.55\.116\.([0-9]|[1-5][0-9]|6[0-3])$|" &_
                  "^65\.55\.111\.(6[4-9]|[7-9][0-9]|1([0-1][0-9]|2[0-7]))$|" &_
                  "^65\.55\.116\.(6[4-9]|[7-9][0-9]|1([0-1][0-9]|2[0-7]))$|" &_
                  "^65\.55\.111\.(1(2[8-9]|[3-8][0-9]|9[0-1]))$|" &_
                  "^65\.55\.34\.([0-9]|[1-5][0-9]|6[0-3])$|" &_
                  "^65\.55\.34\.(6[4-9]|[7-9][0-9]|1([0-1][0-9]|2[0-7]))$|" &_
                  "^65\.55\.34\.(1(2[8-9]|[3-8][0-9]|9[0-1]))$|" &_
                  "^65\.55\.34\.(1(9[2-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^65\.55\.90\.([0-9]|[1-5][0-9]|6[0-3])$|" &_
                  "^65\.55\.90\.(6[4-9]|[7-9][0-9]|1([0-1][0-9]|2[0-7]))$|" &_
                  "^65\.55\.90\.(1(2[8-9]|[3-8][0-9]|9[0-1]))$|" &_
                  "^65\.55\.90\.(1(9[2-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^65\.54\.51\.(6[4-9]|[7-9][0-9]|1([0-1][0-9]|2[0-7]))$|" &_
                  "^65\.54\.61\.(6[4-9]|[7-9][0-9]|1([0-1][0-9]|2[0-7]))$|" &_
                  "^207\.46\.66\.([0-9]|1[0-5])$|" &_
                  "^157\.55\.0\.(1(9[2-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^157\.55\.1\.(1(2[8-9]|[3-8][0-9]|9[0-1]))$|" &_
                  "^157\.55\.2\.([0-9]|[1-5][0-9]|6[0-3])$|" &_
                  "^157\.55\.2\.(6[4-9]|[7-9][0-9]|1([0-1][0-9]|2[0-7]))$"
   case "outlook.com"   
      ' https://technet.microsoft.com/en-us/library/dn163583(v=exchg.150).aspx (2016-06-24)
      oRegEx.Pattern= "^23\.103\.(1(3[2-5]))\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^23\.103\.(1(3[6-9]|4[0-3]))\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^23\.103\.(1(4[4-9]|5[0-9]))\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^23\.103\.(1(9[8-9]))\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^23\.103\.(2(0[0-7]))\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^40\.(9[2-5])\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^40\.107\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^65\.55\.88\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^65\.55\.169\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^94\.245\.120\.(6[4-9]|[7-9][0-9]|1([0-1][0-9]|2[0-7]))$|" &_
                  "^104\.47\.([0-9]|[1-9][0-9]|1([0-1][0-9]|2[0-7]))\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^134\.170\.101\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^134\.170\.140\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^134\.170\.171\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^157\.55\.133\.([0-9]|[1-9][0-9]|1([0-1][0-9]|2[0-7]))$|" &_
                  "^157\.56\.87\.(1(9[2-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^157\.56\.(1(1[0-1]))\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^157\.56\.112\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^157\.56\.116\.([0-9]|[1-9][0-9]|1([0-1][0-9]|2[0-7]))$|" &_
                  "^157\.56\.120\.([0-9]|[1-9][0-9]|1([0-1][0-9]|2[0-7]))$|" &_
                  "^207\.46\.51\.(6[4-9]|[7-9][0-9]|1([0-1][0-9]|2[0-7]))$|" &_
                  "^207\.46\.100\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^207\.46\.108\.([0-9]|[1-9][0-9]|1([0-1][0-9]|2[0-7]))$|" &_
                  "^207\.46\.163\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^213\.199\.154\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^213\.199\.180\.(1(2[8-9]|[3-8][0-9]|9[0-1]))$|" &_
                  "^216\.32\.(1(8[0-1]))\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$"
   case "twitter.com"   
      oRegEx.Pattern= "^199\.16\.(1(5[6-9]))\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^199\.59\.(1(4[8-9]|5[0-1]))\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^8\.25\.(1(9[4-5]))\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^8\.25\.(1(9[6-7]))\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^204\.92\.114\.203$|^204\.92\.114\.(2(0[4-5]))$|^23\.21\.83\.90$"
   case "facebook.com"   
      oRegEx.Pattern= "^69\.63\.179\.25$|^66\.220\.159\.18$|" &_
                  "^69\.63\.178\.(1(2[8-9]|[3-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^69\.63\.184\.([0-9]|[1-9][0-9]|1([0-1][0-9]|2[0-7]))$|" &_
                  "^66\.220\.144\.(1(2[8-9]|[3-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^66\.220\.155\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^69\.171\.232\.([0-9]|[1-9][0-9]|1([0-1][0-9]|2[0-7]))$|" &_
                  "^69\.171\.232\.(1(2[8-9]|[3-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^66\.220\.157\.([0-9]|[1-9][0-9]|1([0-1][0-9]|2[0-7]))$|" &_
                  "^69\.171\.244\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$"         
   case "linkedin.com"   
      oRegEx.Pattern= "^199\.101\.162\.([0-9]|[1-9][0-9]|1([0-1][0-9]|2[0-7]))$|" &_
                  "^108\.174\.3\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^108\.174\.6\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^216\.136\.162\.65$|^199\.101\.161\.130$"
   case "google.com"   
      ' https://support.google.com/a/answer/60764?hl=en
      oRegEx.Pattern= "^64\.18\.([0-9]|1[0-5])\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^64\.233\.(1([6-8][0-9]|9[0-1]))\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^66\.102\.([0-9]|1[0-5])\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^66\.249\.(8[0-9]|9[0-5])\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^72\.14\.(1(9[2-9])|2([0-4][0-9]|5[0-5]))\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^74\.125\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^108\.177\.([8-9]|1[0-5])\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^173\.194\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^207\.126\.(1(4[4-9]|5[0-9]))\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^209\.85\.(1(2[8-9]|[3-9][0-9])|2([0-4][0-9]|5[0-5]))\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^216\.58\.(1(9[2-9])|2([0-1][0-9]|2[0-3]))\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^216\.239\.(3[2-9]|[4-5][0-9]|6[0-3])\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$|" &_
                  "^172\.217\.([0-9]|[1-2][0-9]|3[0-1])\.([0-9]|[1-9][0-9]|1([0-9][0-9])|2([0-4][0-9]|5[0-5]))$"
   Case Else
      oRegEx.Pattern= "^(?:[0-9]{1,3}\.){3}[0-9]{1,3}$"
   End select

   If oRegEx.Test(strIP) Then
   ....
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
RvdH
Senior user
Senior user
Posts: 458
Joined: 2008-06-27 14:42
Location: Netherlands

Re: Sub OnHELO(oClient) progress?

Postby RvdH » 2016-09-13 15:59

Is it safe to block anything that comes from direct IP, eg: 'HELO 172.111.198.131' and/or 'HELO [172.111.198.131]'?

I been running it for a while like below (and BAN them accordingly) and all results in ipranges ban entries contain entries from questionable origin, eg: India, Pakistan, Vietnam, China etc. etc.
I have had zero false entries this far


Code: Select all

      oRegEx.Pattern = "^(\[?" & oClient.IPAddress & "\]?)$|^(\.[a-z]+)$|^(.+\.\.[a-z]+)$|([\!\@\#\$\%\^\&\*\(\)\{\}])"
      If oRegEx.Test(oClient.HELO) Then
         Call AutoBan(oClient.IPAddress, oClient.HELO, 1, "h")
         Result.Message = "Rejected - HELO message should contain FQDN"
         Result.Value = 2
         Exit Sub
      End If
      
      oRegEx.Pattern = "^(\[?(?:[0-9]{1,3}\.){3}[0-9]{1,3}\]?)$"
      If oRegEx.Test(oClient.HELO) Then
         REM local lan IP adresses
         ' oRegEx.Pattern = "^(\[?1((0)|(92\.168)|(72\.((1[6-9])|(2[0-9])|(3[0-1])))|(27))\..*\]?)$"
         ' If oRegEx.Test(oClient.HELO) Then
            ' Result.Value = 0
            ' Exit Sub
         ' Else   
            Call AutoBan(oClient.IPAddress, oClient.HELO, 1, "h")
            Result.Message = "Rejected - HELO message should contain FQDN"
            Result.Value = 2
            Exit Sub
         ' End If
      End If
      


http://www.linuxmagic.com/best_practice ... omain.html
http://faculty.cs.niu.edu/~rickert/cf/bad-ehlo.html
https://github.com/Exim/exim/wiki/AclHeloTricks
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
SorenR
Senior user
Senior user
Posts: 2327
Joined: 2006-08-21 15:38
Location: Denmark

Re: Sub OnHELO(oClient) progress?

Postby SorenR » 2016-09-13 16:19

RFC 2821 (SMTP) says:
"In situations in which the SMTP client system does not have a meaningful domain name (e.g., when its address is dynamically allocated and no reverse mapping record is available), the client SHOULD send an address literal"

So... It's up to you if you want to go "above and beyond" :mrgreen:

Spammers break RFC rules all the time and if they can, so can we ... 8)
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
RvdH
Senior user
Senior user
Posts: 458
Joined: 2008-06-27 14:42
Location: Netherlands

Re: Sub OnHELO(oClient) progress?

Postby RvdH » 2016-09-13 16:24

Yep i noticed that line as well :wink:
in my results it are likely all spammers, considering their questionable origin...guess i'll stick with it!

BTW, the above rules only filter HELO entries on port 25, our clients all use port 587 as SMTP port (example thunderbird uses: HELO [127.0.0.1])
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
SorenR
Senior user
Senior user
Posts: 2327
Joined: 2006-08-21 15:38
Location: Denmark

Re: Sub OnHELO(oClient) progress?

Postby SorenR » 2016-09-13 16:44

RvdH wrote:Yep i noticed that line as well :wink:
in my results it are likely all spammers, considering their questionable origin...guess i'll stick with it!

BTW, the above rules only filter HELO entries on port 25, our clients all use port 587 as SMTP port (example thunderbird uses: HELO [127.0.0.1])

Thunderbird should resolve it's own address with an rDNS lookup, if it fails to do so and there IS a FQDN for the client it must be an old bug in Thunderbird from 2009 that has resurfaced...

https://bugzilla.mozilla.org/show_bug.cgi?id=279525
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
RvdH
Senior user
Senior user
Posts: 458
Joined: 2008-06-27 14:42
Location: Netherlands

Re: Sub OnHELO(oClient) progress?

Postby RvdH » 2016-09-14 15:03

This all got me thinking and makes me wonder if something like a OnAUTH(oClient) event handler could be useful for scripting, eg:

Return String Username
Return Boolean IsAuthenticated
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
SorenR
Senior user
Senior user
Posts: 2327
Joined: 2006-08-21 15:38
Location: Denmark

Re: Sub OnHELO(oClient) progress?

Postby SorenR » 2016-09-14 17:34

RvdH wrote:This all got me thinking and makes me wonder if something like a OnAUTH(oClient) event handler could be useful for scripting, eg:

Return String Username
Return Boolean IsAuthenticated

Hmm... oClient only have 4 members; HELO, IPAddress, Port and Username...

Perhaps two events?

OnAUTH(oClient) and OnAUTHFailed(oClient, sReason)
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
RvdH
Senior user
Senior user
Posts: 458
Joined: 2008-06-27 14:42
Location: Netherlands

Re: Sub OnHELO(oClient) progress?

Postby RvdH » 2016-09-14 17:41

SorenR wrote:
RvdH wrote:This all got me thinking and makes me wonder if something like a OnAUTH(oClient) event handler could be useful for scripting, eg:

Return String Username
Return Boolean IsAuthenticated

Hmm... oClient only have 4 members; HELO, IPAddress, Port and Username...

Perhaps two events?

OnAUTH(oClient) and OnAUTHFailed(oClient, sReason)



Add a member to oClient? :)

Anyway, i'm trying to build something on my dev box...without much luck until now :)
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
RvdH
Senior user
Senior user
Posts: 458
Joined: 2008-06-27 14:42
Location: Netherlands

Re: Sub OnHELO(oClient) progress?

Postby RvdH » 2016-09-16 16:01

Anyone can give me some pointers where to add oClient methods?
So far i have found these items in: InterfaceClient.h, InterfaceClient,cpp, ClientInfo.h, ClientInfo.cpp

Still trying to get something like oClient.Authenticated Boolean value and/or oClient.STARTTLS Boolean value as requested here
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
RvdH
Senior user
Senior user
Posts: 458
Joined: 2008-06-27 14:42
Location: Netherlands

Re: Sub OnHELO(oClient) progress?

Postby RvdH » 2016-10-21 14:05

For the enthusiasts...

hMaiiServer 5.6.6-B2383.3

It contains these 3 fixes
  • Supports Sub OnHELO(oClient) event, issue #153
  • Fixed Incorrect DEBUG logging for event 'OnDeliverMessage', issue #181
  • Include HTMLBody into IMAP TEXT search, pull #193

5.6.6-B2383.3.7z
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
RvdH
Senior user
Senior user
Posts: 458
Joined: 2008-06-27 14:42
Location: Netherlands

Re: Sub OnHELO(oClient) progress?

Postby RvdH » 2016-12-13 18:11

5.6.6-B2383.6

it contains the previous fixes from 5.6.6-B2383.3, plus:

  • Fixed implicit conversion: "int" to "unsigned char" pull #204
  • Faulty: SMTP 'Disconnect client after too many invalid commands' pull issue #160
  • SMTP server error "550 Unsupported ESMTP extension" on MAIL FROM:... AUTH=<> [with fix] issue #164

5.6.6-B2383.6.7z
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
RvdH
Senior user
Senior user
Posts: 458
Joined: 2008-06-27 14:42
Location: Netherlands

Re: Sub OnHELO(oClient) progress?

Postby RvdH » 2017-01-18 16:49

5.6.6-B2383.7

it contains the previous fixes from 5.6.6-B2383.6, plus:

  • Removed warning if backup was more than 1,5GB and 15GB limit. There's no longer a recommended max-size - the time will vary with the installation size. issue #69

5.6.6-B2383.7.7z


Beta builds:

5.6.7-B2405.6

  • Supports Sub OnHELO(oClient) event, issue #153
  • Fixed Incorrect DEBUG logging for event 'OnDeliverMessage', issue #181
  • Include HTMLBody into IMAP TEXT search, pull #193
  • Fixed implicit conversion: "int" to "unsigned char" pull #204
  • Faulty: SMTP 'Disconnect client after too many invalid commands' pull issue #160
  • SMTP server error "550 Unsupported ESMTP extension" on MAIL FROM:... AUTH=<> [with fix] issue #164

5.6.7-B2405.6.7z

5.6.7-B2405.7

it contains the previous fixes from 5.6.7-B2405.6, plus:

  • Removed warning if backup was more than 1,5GB and 15GB limit. There's no longer a recommended max-size - the time will vary with the installation size. issue #69

5.6.7-B2405.7.7z
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
jimimaseye
Moderator
Moderator
Posts: 6402
Joined: 2011-09-08 17:48

Re: Sub OnHELO(oClient) progress?

Postby jimimaseye » 2017-01-18 17:19

Can you clarify something: what is the difference between 5.6.6-b2383.7 and beta (5.6.7-b2405.7) ? Because the changelog mods being shown against both seem to be the same (from 5.6.6-B2383.3). What makes them different/what am I not seeing?
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
RvdH
Senior user
Senior user
Posts: 458
Joined: 2008-06-27 14:42
Location: Netherlands

Re: Sub OnHELO(oClient) progress?

Postby RvdH » 2017-01-18 17:30

Read the beta changelog?

https://www.hmailserver.com/changelog/?version=5.6.7

But in short...

5.6.6-b2383.7 = OpenSSL 1.0.1u
5.6.7-b2405.7 = OpenSSL upgraded from 1.0.1u to 1.0.2j, Upgraded BOOST from 1.56.0 to 1.63.0, https://github.com/hmailserver/hmailserver/issues/208
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
jimimaseye
Moderator
Moderator
Posts: 6402
Joined: 2011-09-08 17:48

Re: Sub OnHELO(oClient) progress?

Postby jimimaseye » 2017-01-18 17:53

Gotcha. Now I understand.

So the BETA versions are Martins genuine betas with your additions (OnHELO etc).
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
RvdH
Senior user
Senior user
Posts: 458
Joined: 2008-06-27 14:42
Location: Netherlands

Re: Sub OnHELO(oClient) progress?

Postby RvdH » 2017-01-18 17:59

Exactly
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
jimimaseye
Moderator
Moderator
Posts: 6402
Joined: 2011-09-08 17:48

Re: Sub OnHELO(oClient) progress?

Postby jimimaseye » 2017-01-18 18:11

Effectively you have brought forward many of the 5.7 fixes ('issues' fixed) into existing 5.6.6 (with your OnHELO addition) for people to use . So people do not have to wait for martin to release 5.7 for these. (Just as well as he seems to be off the boil regarding moving forward with this project, it doesnt seem to be coming forward at any speed - too busy with work etc I presume).
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
jimimaseye
Moderator
Moderator
Posts: 6402
Joined: 2011-09-08 17:48

Re: Sub OnHELO(oClient) progress?

Postby jimimaseye » 2017-02-02 20:42

RvdH wrote:5.6.6-B2383.7

it contains the previous fixes from 5.6.6-B2383.6, plus:
.
.
.

So, Ruud, with your new found Hmailserver coding skills, whats the chances of you making this mod: https://github.com/hmailserver/hmailserver/issues/178 ?

Adding an Autoban is already in the source somewhere, DisableAUTHList is already in the source somewhere, maybe you can work out how to add the autobanning to the DisableAUTHList function?
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
RvdH
Senior user
Senior user
Posts: 458
Joined: 2008-06-27 14:42
Location: Netherlands

Re: Sub OnHELO(oClient) progress?

Postby RvdH » 2017-02-08 22:25

jimimaseye wrote:
RvdH wrote:5.6.6-B2383.7

it contains the previous fixes from 5.6.6-B2383.6, plus:
.
.
.

So, Ruud, with your new found Hmailserver coding skills, whats the chances of you making this mod: https://github.com/hmailserver/hmailserver/issues/178 ?

Adding an Autoban is already in the source somewhere, DisableAUTHList is already in the source somewhere, maybe you can work out how to add the autobanning to the DisableAUTHList function?


I doubt with my "skills" this can be accomplished :wink:

At the time you posted that topic i agreed this could be useful, but now few months later i have to say it seems unnecessary (at least for me) i hardly see login attempts on port 25 anymore...so it seems the abusers/attackers do learn after a while and simply give up trying
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
RvdH
Senior user
Senior user
Posts: 458
Joined: 2008-06-27 14:42
Location: Netherlands

Re: Sub OnHELO(oClient) progress?

Postby RvdH » 2017-02-08 22:58

btw, with the fixes of issue #160 the 'Authentication not enabled' error contributes to the Disconnect client after too many invalid commands counter


Code: Select all

      if (!GetAuthIsEnabled_())
      {
         SendErrorResponse_(504, "Authentication not enabled.");
         return;
      }


Code: Select all

   void
   SMTPConnection::SendErrorResponse_(int iErrorCode, const String &sResponse)
   {
      if (iErrorCode >= 500 && iErrorCode <= 599)
      {
         cur_no_of_invalid_commands_++;

         if (Configuration::Instance()->GetDisconnectInvalidClients() &&
            cur_no_of_invalid_commands_ > Configuration::Instance()->GetMaximumIncorrectCommands())
         {
            // Disconnect
            EnqueueWrite_("Too many invalid commands. Bye!");
            pending_disconnect_ = true;
            EnqueueDisconnect();
            return;
         }
      }

      String sData;
      sData.Format(_T("%d %s"), iErrorCode, sResponse.c_str());

      EnqueueWrite_(sData);
   }



Maybe it could be worth to issue a immediate disconnect after the 'Authentication not enabled' error , eg:

Code: Select all

      if (!GetAuthIsEnabled_())
      {
         SendErrorResponse_(504, "Authentication not enabled.");
         pending_disconnect_ = true;
         EnqueueDisconnect();
         return;
      }
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
RvdH
Senior user
Senior user
Posts: 458
Joined: 2008-06-27 14:42
Location: Netherlands

Re: Sub OnHELO(oClient) progress?

Postby RvdH » 2017-07-01 09:20

5.6.7-B2407.9

This is the latest beta Version 5.6.7 - Build 2407 (2017-01-26)
It contains the previous fixes from 5.6.7-B2405.7, plus:

  • Speed up 'update hm_messages set messageflags' #221
  • Treat authenticated users as localsender if the sender is authenticated and AuthUserIsLocal=1 INI setting Office 2016 Bug


5.6.7-B2407.9.7z
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
Dravion
Senior user
Senior user
Posts: 704
Joined: 2015-09-26 11:50
Location: Germany
Contact:

Re: Sub OnHELO(oClient) progress?

Postby Dravion » 2017-07-01 11:27

What is your plan for this fork in the Future?
Does Martin accept any Pullrequests so this can be merged or backported into the officia release branches on Github?

This was my Problem working on the server core because Martin said verry straight "he gives a fuck what other people are doing, its just its square time project"
My experimental builds of hMailserver 32/64-Bit+Unicode
https://github.com/Dravion/hmailserver/releases

HMSInfo 32/64-Bit
https://github.com/Dravion/HMSInfo/releases

User avatar
SorenR
Senior user
Senior user
Posts: 2327
Joined: 2006-08-21 15:38
Location: Denmark

Re: Sub OnHELO(oClient) progress?

Postby SorenR » 2017-07-01 16:01

Dravion wrote:What is your plan for this fork in the Future?
Does Martin accept any Pullrequests so this can be merged or backported into the officia release branches on Github?

This was my Problem working on the server core because Martin said verry straight "he gives a fuck what other people are doing, its just its square time project"

Well... Ubunto began it's life as a fork from Debian and ...

Jomla is a fork from Mambo
Webkit (by Apple) is a fork from KHTML
WordPress is a fork from b2/CafeLog
Microsoft SQL Server is a fork from Sybase SQL Server
Apache HTTP Server is a fork from NCSA HTTPd

8)

"If you don't like it, fork it."

https://www.theregister.co.uk/2016/07/28/open_source_insider_open_source_forkery/
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
Dravion
Senior user
Senior user
Posts: 704
Joined: 2015-09-26 11:50
Location: Germany
Contact:

Re: Sub OnHELO(oClient) progress?

Postby Dravion » 2017-07-01 18:43

SorenR wrote:
Dravion wrote:What is your plan for this fork in the Future?
Does Martin accept any Pullrequests so this can be merged or backported into the officia release branches on Github?
"If you don't like it, fork it."

I forked it allready in the past. However...
If its not possible collaborating with Martin, it makes no sense for me trying to improve his work and nothing will improve or help the Project. Its fighting against Windmills i think.

For hMailServer i think we can do some extensions and utilities which can be used with hMailServer to. Currently i work with Declan on a Remote API TCP/IP Remote Windows Service. This new Windows Service will add a new TCP/IP Remote Access API Server to hMailServer (for TLS we use LibreSSL). The Remote Access Server for hMailServer can be used to remotely control hMailServer without the need of a Webserver, COM/DCOM and remote clients are not forced running Windows or using a Webbrowser.This Detail is important because Declan wants remotely connect and admistre hMailServer from its own Android Java Smartphone App. So, Java and C/C++ support will be avaiable from the start, but we plan to use LUA as integrated builtin scripting language so non C++ Programmers can remotely script some tasks with LUA which is allmost as easy to learn as VBScript.
My experimental builds of hMailserver 32/64-Bit+Unicode
https://github.com/Dravion/hmailserver/releases

HMSInfo 32/64-Bit
https://github.com/Dravion/HMSInfo/releases

User avatar
RvdH
Senior user
Senior user
Posts: 458
Joined: 2008-06-27 14:42
Location: Netherlands

Re: Sub OnHELO(oClient) progress?

Postby RvdH » 2017-07-03 10:08

I have no idea really...the builds i make only contain fixes that are worth something for myself or fixes that in my opinion already should have been merged into the current 5.6.7 branch
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

AndreL
Normal user
Normal user
Posts: 30
Joined: 2016-06-07 15:42

Re: Sub OnHELO(oClient) progress?

Postby AndreL » 2017-08-03 08:39

A big chunk of my users will migrate to Outlook 2016 (mandatory for integration with another tool).

So I installed the version 5.6.7-B2415.9 from RvdH (previous one was 5.6.7-B2407). So far totally stable.
Annoying bugs like the “office 2016 bug” are solved. Thanks !

Still the missing feature is the support of the Outlook Imap folders auto discovery.
Any advice or guidelines are welcome.

User avatar
RvdH
Senior user
Senior user
Posts: 458
Joined: 2008-06-27 14:42
Location: Netherlands

Re: Sub OnHELO(oClient) progress?

Postby RvdH » 2017-08-05 21:33

5.6.7-B2415.10

This is the latest beta Version 5.6.7 - Build 2415 (2017-07-09)
It contains the previous fixes from 5.6.7-B2405.9, plus:

  • Add Return-Path header as topmost header before sending the message to SA (+ delete Return-Path header after the SA check completes) #116


5.6.7-B2415.10.7z
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
Dravion
Senior user
Senior user
Posts: 704
Joined: 2015-09-26 11:50
Location: Germany
Contact:

Re: Sub OnHELO(oClient) progress?

Postby Dravion » 2017-08-05 23:26

I downloaded your archive but i cannot find the source.
Do you have a Github Repo?
My experimental builds of hMailserver 32/64-Bit+Unicode
https://github.com/Dravion/hmailserver/releases

HMSInfo 32/64-Bit
https://github.com/Dravion/HMSInfo/releases

User avatar
RvdH
Senior user
Senior user
Posts: 458
Joined: 2008-06-27 14:42
Location: Netherlands

Re: Sub OnHELO(oClient) progress?

Postby RvdH » 2017-08-06 10:31

Dravion wrote:I downloaded your archive but i cannot find the source.
Do you have a Github Repo?


Now i do... :lol:
https://github.com/RvdHout/hmailserver

i committed all my changes at once...so you have to look carefully what is actually changed
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
Dravion
Senior user
Senior user
Posts: 704
Joined: 2015-09-26 11:50
Location: Germany
Contact:

Re: Sub OnHELO(oClient) progress?

Postby Dravion » 2017-08-06 10:44

Cool. :D
I will take a look at it.

ps:
As far as can see you made changes in this
section of the code only, is this correct?

https://github.com/RvdHout/hmailserver/ ... erver/SMTP
My experimental builds of hMailserver 32/64-Bit+Unicode
https://github.com/Dravion/hmailserver/releases

HMSInfo 32/64-Bit
https://github.com/Dravion/HMSInfo/releases

User avatar
RvdH
Senior user
Senior user
Posts: 458
Joined: 2008-06-27 14:42
Location: Netherlands

Re: Sub OnHELO(oClient) progress?

Postby RvdH » 2017-08-06 11:04

  • hmailserver/source/Server/Common/AntiSpam/SpamTestSpamAssassin.cpp
  • hmailserver/source/Server/Common/Application/BackupExecuter.cpp
  • hmailserver/source/Server/Common/Application/IniFileSettings.cpp
  • hmailserver/source/Server/Common/Application/IniFileSettings.h
  • hmailserver/source/Server/Common/BO/Messages.cpp
  • hmailserver/source/Server/Common/Persistence/PersistentMessage.cpp
  • hmailserver/source/Server/Common/Persistence/PersistentMessage.h
  • hmailserver/source/Server/Common/Scripting/Events.cpp
  • hmailserver/source/Server/Common/Scripting/ScriptServer.cpp
  • hmailserver/source/Server/Common/Scripting/ScriptServer.h
  • hmailserver/source/Server/Common/TCPIP/IPAddress.cpp
  • hmailserver/source/Server/Common/Util/File.cpp
  • hmailserver/source/Server/IMAP/IMAPCommandSearch.cpp
  • hmailserver/source/Server/SMTP/RecipientParser.cpp
  • hmailserver/source/Server/SMTP/SMTPConnection.cpp

And 'Version.h' to reflect the current build nr.
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
RvdH
Senior user
Senior user
Posts: 458
Joined: 2008-06-27 14:42
Location: Netherlands

Re: Sub OnHELO(oClient) progress?

Postby RvdH » 2017-09-19 09:33

5.6.7-B2415.11

This is the latest beta Version 5.6.7 - Build 2415 (2017-07-09)
It contains the previous fixes/additions from 5.6.7-B2405.10, plus:

  • Experimental eventhandler OnClientLogon(oClient), New ClientInfo property oClient.Authenticated (Boolean)

Note:
  • In the OnClientLogon(oClient) event, oClient.Username always holds the value passed when authenticating the user, in later events like OnSmtpData, OnAcceptMessage the oClient.Username is empty when authentication has failed (to be compatible with current behavior/scripts)
  • OnSmtpData, OnAcceptMessage events can also make use of the value oClient.Authenticated (Boolean)

Code: Select all

Sub OnClientLogon(oClient)
   If oClient.Authenticated then
      EventLog.Write("Successful login for " & oClient.Username & " from " & oClient.IpAddress & " on port " & oClient.Port & "")
   Else
      EventLog.Write("Failed login for " & oClient.Username & " from " & oClient.IpAddress & " on port " & oClient.Port & "")
   End if
End Sub


Source:
github.com/RvdHout/hmailserver

Download:
5.6.7-B2415.11.7z
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
mattg
Moderator
Moderator
Posts: 17596
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Sub OnHELO(oClient) progress?

Postby mattg » 2017-09-25 11:23

I've been using this for over a week - looks good

however...
I have thunderbird open all day with some 14 mail accounts connected to my server.

My errors@example.com gets sent automated errors from various systems and scripts etc, and today there is a device that is sending that address an email every five minutes or so.
No big deal, just need to sort this device out

In my event log I get

Code: Select all

2380   "2017-09-25 16:49:07.533"   "Successful login for errors@example.com from 192.168.0.1 on port 143"
2380   "2017-09-25 16:49:07.689"   "Successful login for errors@example.com from 192.168.0.1 on port 143"
8040   "2017-09-25 16:49:07.845"   "Successful login for errors@example.com from 192.168.0.1 on port 143"
7360   "2017-09-25 16:49:08.017"   "Successful login for errors@example.com from 192.168.0.1 on port 143"
7360   "2017-09-25 16:49:08.173"   "Successful login for errors@example.com from 192.168.0.1 on port 143"
2992   "2017-09-25 16:49:08.349"   "Successful login for errors@example.com from 192.168.0.1 on port 143"
8040   "2017-09-25 16:49:08.517"   "Successful login for errors@example.com from 192.168.0.1 on port 143"
8996   "2017-09-25 16:49:08.879"   "Successful login for errors@example.com from 192.168.0.1 on port 143"
2380   "2017-09-25 16:49:09.048"   "Successful login for errors@example.com from 192.168.0.1 on port 143"
5052   "2017-09-25 16:49:09.580"   "Successful login for errors@example.com from 192.168.0.1 on port 143"
8996   "2017-09-25 16:49:10.298"   "Successful login for errors@example.com from 192.168.0.1 on port 143"


None of MY other accounts get logged
This account is the only account that has new mail
There is only 11 lines, not 14
This repeats every five minutes as the new email to example.com arrives
All accounts are set for IMAP IDLE

None of this is really important, just thought you'd like to know

Every time I send an email, it registers as logging on, and everytime I receive an email I get multiple entries as above.
Someone logging on via POP3 gets logged as expected, all outgoing mail is logged, I guess as expected, just the IMAP accounts when receiving new mail.
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
RvdH
Senior user
Senior user
Posts: 458
Joined: 2008-06-27 14:42
Location: Netherlands

Re: Sub OnHELO(oClient) progress?

Postby RvdH » 2017-09-27 12:10

I think this behaviour is correct...
If you enable debug logging you'lll notice the IMAP client sends the 'A0002 LOGIN' command when fetching new messages, for example roundcube does this with every IMAP command send to the server
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
mattg
Moderator
Moderator
Posts: 17596
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Sub OnHELO(oClient) progress?

Postby mattg » 2017-10-08 02:30

Because I have been using this build I also now have access to Sub OnHELO

This is heaps of fun.
swap 'XXX.XXX.XXX.XXX' for your public IP address

I use the random rejection messages in a few other places in my eventhandlers.vbs now. Thanks for the idea SorenR.
Not that it matters, because I don't think that anyone is listening to my witty auto-replies.

Code: Select all

Sub OnHELO(oClient)
   If oClient.helo = "XXX.XXX.XXX.XXX" Or oClient.helo = "[XXX.XXX.XXX.XXX]"  Or oClient.helo = "ylmf-pc" Or oClient.helo = "User" Then
      Result.Message = RandomRejection()
      EventLog.Write("Random Rejection sent to IP - " & oClient.IPAddress & "  Rejection Message was '" & Result.Message & "'")
      Result.Value = 2
   End If   
End Sub

Function RandomRejection()
   Dim i, RejectionStrings(10)
   RejectionStrings(0) = "I'm sorry Jim, I can't do that"
   RejectionStrings(1) = "Hasta la vista, Baby"
   RejectionStrings(2) = "Have you tried switching it off and then on again"
   RejectionStrings(3) = "It's hard to have a battle of wits with someone who is half armed"
   RejectionStrings(4) = "Computer says 'No'"
   RejectionStrings(5) = "Artificial intelligence is no match for natural stupidity"
   RejectionStrings(6) = "Oooops LOL"
   RejectionStrings(7) = "Here comes Karma..."
   RejectionStrings(8) = "How did that happen?"
   RejectionStrings(9) = "Now go away or I shall taunt you again"
   RejectionStrings(10) = "The last thing I want to do is insult you. But it IS on the list."
   Randomize
   i = int(Rnd()*11)
   RandomRejection = RejectionStrings(i)
End Function
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
RvdH
Senior user
Senior user
Posts: 458
Joined: 2008-06-27 14:42
Location: Netherlands

Re: Sub OnHELO(oClient) progress?

Postby RvdH » 2017-10-08 17:29

Nice one :lol:

Btw, have you checked IMAP logging (i wrote debug before i see now) and do you agree the behavior resulting in the excessive 'Successful login' logging is caused by client sending LOGIN commands?
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
mattg
Moderator
Moderator
Posts: 17596
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Sub OnHELO(oClient) progress?

Postby mattg » 2017-10-25 05:26

I've noticed that a few times my hMailserver has simply stopped accepting new connections, and I need to restart via Windows Services.

This seems to stop after something happens in my OnHelo sub

"SMTPD" 6692 2922 "2017-10-25 11:34:08.364" "203.113.204.51" "RECEIVED: EHLO User"
"DEBUG" 6692 "2017-10-25 11:34:08.364" "Executing event OnHELO"
"DEBUG" 3756 "2017-10-25 11:34:34.118" "No messages to index."
"DEBUG" 3756 "2017-10-25 11:35:36.489" "No messages to index."
"DEBUG" 3756 "2017-10-25 11:36:38.849" "No messages to index."
"DEBUG" 3756 "2017-10-25 11:37:41.208" "No messages to index."
"DEBUG" 3756 "2017-10-25 11:38:43.568" "No messages to index."
"DEBUG" 3756 "2017-10-25 11:39:45.958" "No messages to index."
"DEBUG" 3756 "2017-10-25 11:40:48.333" "No messages to index."
"DEBUG" 3756 "2017-10-25 11:41:50.701" "No messages to index."

The only thing that continues to work is External account download, nothing else...until I restart the service

Code: Select all

Sub OnHELO(oClient)
   Dim tempString
   EventLog.Write("Details for OnHELO - IP Address = " & oClient.IPAddress & "  Helo sent was '" & CStr(oClient.helo) & "'")
   If oClient.helo = "ylmf-pc" Or oClient.helo = "User" Then
       TempString = "    Helo = '" & oClient.helo & "' was Autobanned for using known spammer EHLO"
      Call CustomMonthlyLog(TempString, "Port" & oClient.port)
      Call AutobanIP(oClient.ipaddress, 14, "Using My IP as EHLO")
      Result.Message = RandomRejection()
      EventLog.Write("Random Rejection sent to IP - " & oClient.IPAddress & "  Rejection Message was '" & Result.Message & "'")
       Wait(14)
      Result.Value = 2
   End If   
End Sub


This was written to my events log
6692 "2017-10-25 11:34:08.364" "Details for OnHELO - IP Address = 203.113.204.51 Helo sent was 'User'"

This was written to my port 25 log (this connection was port 25)
2017-10-25 11:34:08.394 Helo = 'User' was Autobanned for using known spammer EHLO

But then no Autoban happened.

Obviously ends unexpectedly somewhere in my AutoBanIP sub.
I've just added some eventlog lines to my AutoBanIP sub to try and find where it stops

Here is what it currently looks like, and it has been banning ~15/25 per day from elsewhere in my eventhandlers.vbs, mostly overseas connections to IMAP or POP3 ports

Code: Select all

Sub AutobanIP(IPAddress, NumberOfDays, ReasonForBan)
   EventLog.Write("Autoban IP Address started for IP = " & IPAddress & " For " & NumberofDays & " Number of days for reason " & Reasonforban)
   Dim oApp
   Set oApp = CreateObject("hMailServer.Application")

'       Give this script permission to access all
'       hMailServer settings.
   Call oApp.Authenticate("Administrator", g_sAdminPassword)

   Dim oSecurityRange, i
   For i = 0 To oApp.Settings.SecurityRanges.Count -1
      If IPAddress = oApp.Settings.SecurityRanges.Item(i).LowerIP Then Exit sub
   Next
   Set oSecurityRange = oApp.Settings.SecurityRanges.Add()
   EventLog.Write("Autoban IP range being set for IP Address " & IPAddress)
   With oSecurityRange
      .lowerip = ipaddress
      .upperip = ipaddress
      .priority = 20
      .allowdeliveryfromlocaltolocal = False
      .allowdeliveryfromlocaltoremote = False
      .allowdeliveryfromremotetolocal = False
      .allowdeliveryfromremotetoremote = False
      .allowimapconnections = False
      .allowsmtpconnections = False
      .allowpop3connections = False
      .expires = True
      .ExpiresTime = DateAdd("d", NumberOfDays, Now())
      .name = ReasonForBan & " - banned for " & NumberOfDays & " days"
'         .name = "script " & IPAddress & " - banned for " & NumberOfDays & " days - Reason is " & ReasonForBan
      i = False
      While i = False
         On Error Resume Next
         .save
         If Err.Number = 0 Then
            EventLog.Write("Autoban IP range saved for IP Address " & IPAddress)
            On Error Goto 0
            i = True
            Exit Sub
         End If
         Wait(1)
      Wend
   End With
End Sub
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
mattg
Moderator
Moderator
Posts: 17596
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Sub OnHELO(oClient) progress?

Postby mattg » 2017-10-25 05:44

Just had another (whilst typing this)

"DEBUG" 7516 "2017-10-25 13:21:41.112" "TCP connection started for session 79"
"SMTPD" 7516 79 "2017-10-25 13:21:41.112" "203.113.204.51" "SENT: 220 example.com ESMTP"
"SMTPD" 2588 79 "2017-10-25 13:21:41.144" "203.113.204.51" "RECEIVED: EHLO User"
"DEBUG" 2588 "2017-10-25 13:21:41.144" "Executing event OnHELO"
"DEBUG" 7512 "2017-10-25 13:22:13.472" "No messages to index."
"DEBUG" 7512 "2017-10-25 13:23:15.836" "No messages to index."
"DEBUG" 7512 "2017-10-25 13:24:18.206" "No messages to index."

"DEBUG" 8044 "2017-10-25 13:26:41.145" "The client has timed out. Session: 79"
"SMTPD" 8044 79 "2017-10-25 13:26:41.145" "203.113.204.51" "SENT: 421 Connection timeout."
"DEBUG" 8772 "2017-10-25 13:26:46.159" "The client has timed out. Session: 79"


My eventlog says
2588 "2017-10-25 13:21:41.191" "Autoban IP Address started for IP = 203.113.204.51 For 14 Number of days for reason Using My IP as EHLO"
2588 "2017-10-25 13:21:41.300" "Autoban IP range being set for IP Address 203.113.204.51"


The save didn't happen...?

But no Autoban created (I've done it manually now)
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
RvdH
Senior user
Senior user
Posts: 458
Joined: 2008-06-27 14:42
Location: Netherlands

Re: Sub OnHELO(oClient) progress?

Postby RvdH » 2017-10-25 10:05

Can't say i have experienced something like this...or at least i did not notice it


I use SorenR's scripts to autoban

Code: Select all

   Function Wait(sec)
      With CreateObject("WScript.Shell")
         .Run "timeout /T " & Int(sec), 0, True
'        .Run "sleep -m " & Int(sec * 1000), 0, True
'        .Run "powershell Start-Sleep -Milliseconds " & Int(sec * 1000), 0, True
      End With
   End Function

   Function LockFile(strPath)
      Const Append = 8
      Const Unicode = -1
      With CreateObject("Scripting.FileSystemObject")
         Dim oFile, i
         For i = 0 To 30
            On Error Resume Next
            Set oFile = .OpenTextFile(strPath, Append, True, Unicode)
            If (Not Err.Number = 70) Then
               Set LockFile = oFile
               On Error Goto 0
               Exit For
            End If
            On Error Goto 0
            Wait(1)
         Next
      End With
      Set oFile = Nothing
      If (Err.Number = 70) Then
         EventLog.Write("ERROR: EventHandlers.vbs")
         EventLog.Write("File " & strPath & " is locked and timeout was exceeded.")
         Err.Clear
      ElseIf (Err.Number <> 0) Then
         EventLog.Write("ERROR: EventHandlers.vbs : Function LockFile")
         EventLog.Write("Error       : " & Err.Number)
         EventLog.Write("Error (hex) : 0x" & Hex(Err.Number))
         EventLog.Write("Source      : " & Err.Source)
         EventLog.Write("Description : " & Err.Description)
         Err.Clear
      End If
   End Function

   Function Lookup(strRegEx, strMatch)
      With CreateObject("VBScript.RegExp")
         .Global = False
         .Pattern = strRegEx
         .IgnoreCase = True
         If .Test(strMatch) Then
            Lookup = True
         Else
            Lookup = False
         End If
      End With
   End Function

   '
   ' sType can be one of the following;
   ' "yyyy" Year, "m" Month, "d" Day, "h" Hour, "n" Minute, "s" Second
   '
   Sub AutoBan(sIPAddress, sReason, iDuration, sType)
      Dim oApp : Set oApp = CreateObject("hMailServer.Application")
      Call oApp.Authenticate(ADMIN, PASSWORD)
      With LockFile("c:\hmailserver\temp\autoban.lck")
         On Error Resume Next
         oApp.Settings.SecurityRanges.Refresh
         If (oApp.Settings.SecurityRanges.ItemByName("(" & sReason & ") " & sIPAddress) Is Nothing) Then
            With oApp.Settings.SecurityRanges.Add
               .Name = "(" & sReason & ") " & IPAddress
               .LowerIP = sIPAddress
               .UpperIP = sIPAddress
               .Priority = 20
               .Expires = True
               .ExpiresTime = DateAdd(sType, iDuration, Now())
               .Save
            End With
         End If
         On Error Goto 0
         .Close
      End With
   End Sub
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
SorenR
Senior user
Senior user
Posts: 2327
Joined: 2006-08-21 15:38
Location: Denmark

Re: Sub OnHELO(oClient) progress?

Postby SorenR » 2017-10-26 01:15

mattg wrote:Just had another (whilst typing this)

"DEBUG" 7516 "2017-10-25 13:21:41.112" "TCP connection started for session 79"
"SMTPD" 7516 79 "2017-10-25 13:21:41.112" "203.113.204.51" "SENT: 220 example.com ESMTP"
"SMTPD" 2588 79 "2017-10-25 13:21:41.144" "203.113.204.51" "RECEIVED: EHLO User"
"DEBUG" 2588 "2017-10-25 13:21:41.144" "Executing event OnHELO"
"DEBUG" 7512 "2017-10-25 13:22:13.472" "No messages to index."
"DEBUG" 7512 "2017-10-25 13:23:15.836" "No messages to index."
"DEBUG" 7512 "2017-10-25 13:24:18.206" "No messages to index."

"DEBUG" 8044 "2017-10-25 13:26:41.145" "The client has timed out. Session: 79"
"SMTPD" 8044 79 "2017-10-25 13:26:41.145" "203.113.204.51" "SENT: 421 Connection timeout."
"DEBUG" 8772 "2017-10-25 13:26:46.159" "The client has timed out. Session: 79"


My eventlog says
2588 "2017-10-25 13:21:41.191" "Autoban IP Address started for IP = 203.113.204.51 For 14 Number of days for reason Using My IP as EHLO"
2588 "2017-10-25 13:21:41.300" "Autoban IP range being set for IP Address 203.113.204.51"


The save didn't happen...?

But no Autoban created (I've done it manually now)

If you receive concurrent connections (winthin milliseconds) from the same IP and you try to ban that IP then "i" will never become "true" because "Err.Number" can never become "0" ... The IP was banned from another session during the while...wend loop.

I think... :mrgreen:
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
jimimaseye
Moderator
Moderator
Posts: 6402
Joined: 2011-09-08 17:48

Re: Sub OnHELO(oClient) progress?

Postby jimimaseye » 2017-10-26 08:35

SorenR wrote:If you receive concurrent connections (winthin milliseconds) from the same IP and you try to ban that IP then "i" will never become "true" because "Err.Number" can never become "0" ... The IP was banned from another session during the while...wend loop.

I think... :mrgreen:

So how do we stop this situation? Having the server freeze and require restarting is not very practical (to say the least).
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
mattg
Moderator
Moderator
Posts: 17596
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Sub OnHELO(oClient) progress?

Postby mattg » 2017-10-26 09:30

I dim the i in the sub, it should only have scope for the sub, in the current iteration (run) of the sub.

I've changed to use the lockfile like SorenR does, so we will see if that does better...
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
SorenR
Senior user
Senior user
Posts: 2327
Joined: 2006-08-21 15:38
Location: Denmark

Re: RE: Re: Sub OnHELO(oClient) progress?

Postby SorenR » 2017-10-26 13:03

jimimaseye wrote:
SorenR wrote:If you receive concurrent connections (winthin milliseconds) from the same IP and you try to ban that IP then "i" will never become "true" because "Err.Number" can never become "0" ... The IP was banned from another session during the while...wend loop.

I think... :mrgreen:

So how do we stop this situation? Having the server freeze and require restarting is not very practical (to say the least).
That's why I did the filelocking thingy, to control when what is done. A form of forced preemptive multitasking. ;-)
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
jimimaseye
Moderator
Moderator
Posts: 6402
Joined: 2011-09-08 17:48

Re: RE: Re: Sub OnHELO(oClient) progress?

Postby jimimaseye » 2017-10-26 13:45

SorenR wrote:So how do we stop this situation? Having the server freeze and require restarting is not very practical (to say the least).
That's why I did the filelocking thingy, to control when what is done. A form of forced preemptive multitasking. ;-)

Right. I didnt realise matt had not included the filelock-check function.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
mattg
Moderator
Moderator
Posts: 17596
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Sub OnHELO(oClient) progress?

Postby mattg » 2017-10-26 23:20

Stopped again on me overnight
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

mikedibella
Normal user
Normal user
Posts: 59
Joined: 2016-12-08 02:21

Re: Sub OnHELO(oClient) progress?

Postby mikedibella » 2017-10-26 23:53

If you assume you are entering the race condition because the .Save fails due to the presence of a duplicate Autoban entry created on a different thread, wouldn't you see that entry in the database when you restart the service?

If the .Save is failing because a duplicate exists, you need to catch the particular error state in your code and exit from the loop.

I'm not sure I even understand why you think you need to retry Saving indefinitely. Why not just try once and log a failure on error?

User avatar
mattg
Moderator
Moderator
Posts: 17596
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Sub OnHELO(oClient) progress?

Postby mattg » 2017-10-26 23:58

NO Autoban entry exists for that IP, before or after server restart, so it simply isn't created

I will play again with my script and try and catch better errors I think
When my hMailsevrer stopped last night it wasn't due to this OnHELO sub stopping part way through, so I need to investigate what exactly is happening.

I never had the stop issue on the hMailsevrer official Beta build, only since I moved to RVHD's special build. I may need to move back to the official builds too.
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

mikedibella
Normal user
Normal user
Posts: 59
Joined: 2016-12-08 02:21

Re: Sub OnHELO(oClient) progress?

Postby mikedibella » 2017-10-27 00:59

I think you will need to find some usable terminal condition to cause an exit from the loop, because if you get into the loop in a state where the .Save fails, you you will loop endlessly and that is probably what is cause the behavior you are seeing.

insomniac2k2
Normal user
Normal user
Posts: 36
Joined: 2016-08-09 19:47

Re: Sub OnHELO(oClient) progress?

Postby insomniac2k2 » 2017-10-29 19:56

While testing some code that i just implemented for GreyWhitelisting, i ran into an interesting find. This issue is consistent and can be replicated on demand:

Using SorenR's regex logic OnHELO I get a proper match from google and successfully call for a Greylist Whitelist addition. But the interesting part is that I seem to get 2 matches and call for an addition for the same IP within milliseconds of each other. This causes an access violation and throws about 30 errors in the log.

Here's what i can confirm:

It's not my executable that is causing the issue, as i am logging the matches from the EventHandler and writing to the event log before the call. See below:

Code: Select all

   Sub OnHELO(oClient)
   '
   ' BEGIN
   '

   Dim strRegEx
   strRegEx = "^[a-z]+[0-9]{2}(-)[a-z]{2}[0-9](-obe\.outbound\.protection\.outlook\.com)$|"     &_
              "^(a)[0-9]{1,2}(-)[0-9]{1,3}(\.smtp-out\.amazonses\.com)$|"                       &_
              "^(mail-)[a-z]{2}[0-9](-f)[0-9]{1,3}(\.google\.com)$|"                            &_
              "^(spring-chicken-)[a-z]{2}(\.twitter\.com)$|"                                    &_
              "^(mail)[a-z](-)[a-z]{2}(\.linkedin\.com)$|"                                      &_
              "^(mx)[0-9](\.)[a-z]{3}(\.paypal\.com)$|"                                         &_
              "^(mx-out\.facebook\.com)$|"                                                      &_
              "^(relay99\.mysmtp\.com)$|"                                                       &_
              "(\.lifewavemembers\.com)$|"                                                      &_
              "(\.companymobile\.dk)$|"                                                         &_
              "(\.gratisdns\.dk)$|"                                                             &_
              "(\.electric\.net)$|"                                                             &_
              "(\.jobindex\.dk)$|"                                                              &_
              "(\.anpdm\.com)$|"                                                                &_
              "(\.exigo\.com)$|"                                                                &_
              "(\.post\.dk)$"
   If Lookup(strRegEx, oClient.HELO) Then
   EventLog.Write("Message from: " & oClient.IPAddress & " " & oClient.HELO & " Added as to Greylist Whitelist do to match in approved list")
        Set oShell = CreateObject("WScript.Shell")
   oShell.Run """C:\Program Files (x86)\hMailServer\Bin\ProxyAuth.exe"" -secret Mangos -greywhitelist " & oClient.IPAddress & " " & oClient.HELO
      Exit Sub
   End If

   '
   ' END
   '
   End Sub


The google ip address that is being added is this. Notice the redundant entries that happen within milliseconds of each other.:
6060 "2017-10-29 10:39:15.920" "Message from: 74.125.82.50 mail-wm0-f50.google.com Added as to Greylist Whitelist do to match in approved list"
6560 "2017-10-29 10:39:16.639" "Message from: 74.125.82.50 mail-wm0-f50.google.com Added as to Greylist Whitelist do to match in approved list"

This throws the DB error:

"ERROR" 1820 "2017-10-29 10:20:14.588" "Severity: 2 (High), Code: HM5032, Source: DALConnection::Execute, Description: Source: SQLCEConnection::Execute(), Code: HM10044, Description: Error while executing SQL statement:
INSERT INTO hm_greylisting_whiteaddresses (whiteipaddress, whiteipdescription) VALUES (@whiteipaddress2, @whiteipdescription3)
Microsoft SQL Server Compact OLE DB Provider
Value violated the integrity constraints for a column or table."
"ERROR" 5048 "2017-10-29 10:23:29.850" "Severity: 2 (High), Code: HM5032, Source: DALConnection::Execute, Description: Source: SQLCEConnection::Execute(), Code: HM10044, Description: Error while executing SQL statement:
INSERT INTO hm_greylisting_whiteaddresses (whiteipaddress, whiteipdescription) VALUES (@whiteipaddress2, @whiteipdescription3)
Microsoft SQL Server Compact OLE DB Provider
Value violated the integrity constraints for a column or table."

So here is my first guess to what could cause this behavior without cracking the source open:

Is a .tmp file created before OnHELO? If so, it may be possible that this is causing a redundant check against the host, thus causing duplicate matches and duplicate calls? I still have to debug more, but wanted to post my findings in case someone has seen this behavior with OnHELO already.


Actually, reading SorenR's comment above, the issue looks VERY similar.

User avatar
SorenR
Senior user
Senior user
Posts: 2327
Joined: 2006-08-21 15:38
Location: Denmark

Re: RE: Re: Sub OnHELO(oClient) progress?

Postby SorenR » 2017-10-29 23:15

insomniac2k2 wrote:While testing some code that i just implemented for GreyWhitelisting, i ran into an interesting find. This issue is consistent and can be replicated on demand:

Using SorenR's regex logic OnHELO I get a proper match from google and successfully call for a Greylist Whitelist addition. But the interesting part is that I seem to get 2 matches and call for an addition for the same IP within milliseconds of each other. This causes an access violation and throws about 30 errors in the log.

Here's what i can confirm:

It's not my executable that is causing the issue, as i am logging the matches from the EventHandler and writing to the event log before the call. See below:

Code: Select all

   Sub OnHELO(oClient)
   '
   ' BEGIN
   '

   Dim strRegEx
   strRegEx = "^[a-z]+[0-9]{2}(-)[a-z]{2}[0-9](-obe\.outbound\.protection\.outlook\.com)$|"     &_
              "^(a)[0-9]{1,2}(-)[0-9]{1,3}(\.smtp-out\.amazonses\.com)$|"                       &_
              "^(mail-)[a-z]{2}[0-9](-f)[0-9]{1,3}(\.google\.com)$|"                            &_
              "^(spring-chicken-)[a-z]{2}(\.twitter\.com)$|"                                    &_
              "^(mail)[a-z](-)[a-z]{2}(\.linkedin\.com)$|"                                      &_
              "^(mx)[0-9](\.)[a-z]{3}(\.paypal\.com)$|"                                         &_
              "^(mx-out\.facebook\.com)$|"                                                      &_
              "^(relay99\.mysmtp\.com)$|"                                                       &_
              "(\.lifewavemembers\.com)$|"                                                      &_
              "(\.companymobile\.dk)$|"                                                         &_
              "(\.gratisdns\.dk)$|"                                                             &_
              "(\.electric\.net)$|"                                                             &_
              "(\.jobindex\.dk)$|"                                                              &_
              "(\.anpdm\.com)$|"                                                                &_
              "(\.exigo\.com)$|"                                                                &_
              "(\.post\.dk)$"
   If Lookup(strRegEx, oClient.HELO) Then
   EventLog.Write("Message from: " & oClient.IPAddress & " " & oClient.HELO & " Added as to Greylist Whitelist do to match in approved list")
        Set oShell = CreateObject("WScript.Shell")
   oShell.Run """C:\Program Files (x86)\hMailServer\Bin\ProxyAuth.exe"" -secret Mangos -greywhitelist " & oClient.IPAddress & " " & oClient.HELO
      Exit Sub
   End If

   '
   ' END
   '
   End Sub


The google ip address that is being added is this. Notice the redundant entries that happen within milliseconds of each other.:
6060 "2017-10-29 10:39:15.920" "Message from: 74.125.82.50 mail-wm0-f50.google.com Added as to Greylist Whitelist do to match in approved list"
6560 "2017-10-29 10:39:16.639" "Message from: 74.125.82.50 mail-wm0-f50.google.com Added as to Greylist Whitelist do to match in approved list"

This throws the DB error:

"ERROR" 1820 "2017-10-29 10:20:14.588" "Severity: 2 (High), Code: HM5032, Source: DALConnection::Execute, Description: Source: SQLCEConnection::Execute(), Code: HM10044, Description: Error while executing SQL statement:
INSERT INTO hm_greylisting_whiteaddresses (whiteipaddress, whiteipdescription) VALUES (@whiteipaddress2, @whiteipdescription3)
Microsoft SQL Server Compact OLE DB Provider
Value violated the integrity constraints for a column or table."
"ERROR" 5048 "2017-10-29 10:23:29.850" "Severity: 2 (High), Code: HM5032, Source: DALConnection::Execute, Description: Source: SQLCEConnection::Execute(), Code: HM10044, Description: Error while executing SQL statement:
INSERT INTO hm_greylisting_whiteaddresses (whiteipaddress, whiteipdescription) VALUES (@whiteipaddress2, @whiteipdescription3)
Microsoft SQL Server Compact OLE DB Provider
Value violated the integrity constraints for a column or table."

So here is my first guess to what could cause this behavior without cracking the source open:

Is a .tmp file created before OnHELO? If so, it may be possible that this is causing a redundant check against the host, thus causing duplicate matches and duplicate calls? I still have to debug more, but wanted to post my findings in case someone has seen this behavior with OnHELO already.


Actually, reading SorenR's comment above, the issue looks VERY similar.
If you look back in your other thread you'll see in my GreyWhiteList code i am using my eventlocking to avoid violating the database constraint. ;-)
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.


Return to “Development & alpha discussions”



Who is online

Users browsing this forum: No registered users and 4 guests