Sub OnHELO(oClient) progress?

Use this forum if you want to discuss a problem or ask a question related to a hMailServer beta release.
Kelden
Normal user
Normal user
Posts: 46
Joined: 2011-05-02 13:58

Re: Sub OnHELO(oClient) progress?

Post by Kelden » 2021-04-03 01:04

@RvdH
I like most improvements, what I still miss is a whitelist for domains that ignore graylisting.
e.g. *.outbound.protection.outlook.com
There are some scripts, but it's not a nice solution.
Could you add such an option?

User avatar
RvdH
Senior user
Senior user
Posts: 1466
Joined: 2008-06-27 14:42
Location: Netherlands

Re: Sub OnHELO(oClient) progress?

Post by RvdH » 2021-04-03 11:57

Kelden wrote:
2021-04-03 01:04
@RvdH
I like most improvements, what I still miss is a whitelist for domains that ignore graylisting.
e.g. *.outbound.protection.outlook.com
There are some scripts, but it's not a nice solution.
Could you add such an option?
It sounds like you would like to greylistwhitelist connection based on HELO/EHLO *.outbound.protection.outlook.com?
You do know spammers do fake HELO/EHLO headers, right?

So you either have to verify the IP is actually allowed to send from *.outbound.protection.outlook.com using the SPF record or that the PTR matches for that IP (i believe SorenR uses the later, right?)
As there are script solutions available i do not feel the need to incorporate such functionality, not to mention the difficulty incorporating this would involve

The (script) approach below does not only verify the SPF check but it also verifies the received header so it matches the domain it supposed to be arriving from.

As mail arrives from *.outbound.protection.outlook.com is always send using STARTTLS:
On the first event OnHELO (red) the server (might) not be whitelisted, so we check the IP against SPF for the HEL0/EHLO header and whitelist it as needed
On the second event OnHELO (green) after STARTTLS command is inquired the server should be whitelisted from greylisting
the flow is like this:
"TCPIP" 8864 "2021-04-01 09:21:11.235" "TCP - 40.92.89.36 connected to 192.168.2.2:25."
"DEBUG" 8864 "2021-04-01 09:21:11.235" "Executing event OnClientConnect"
"DEBUG" 8864 "2021-04-01 09:21:11.370" "Event completed"
"DEBUG" 8864 "2021-04-01 09:21:11.370" "TCP connection started for session 7"
"SMTPD" 8864 7 "2021-04-01 09:21:11.370" "40.92.89.36" "SENT: 220 mail.server.com ESMTP"
"SMTPD" 7808 7 "2021-04-01 09:21:11.386" "40.92.89.36" "RECEIVED: EHLO EUR05-DB8-obe.outbound.protection.outlook.com"
"DEBUG" 7808 "2021-04-01 09:21:11.386" "Executing event OnHELO"
"DEBUG" 7808 "2021-04-01 09:21:14.039" "Event completed"

"SMTPD" 7808 7 "2021-04-01 09:21:14.039" "40.92.89.36" "SENT: 250-mail.server.com[nl]250-SIZE 51200000[nl]250-STARTTLS[nl]250 HELP"
"SMTPD" 4592 7 "2021-04-01 09:21:14.070" "40.92.89.36" "RECEIVED: STARTTLS"
"SMTPD" 4592 7 "2021-04-01 09:21:14.070" "40.92.89.36" "SENT: 220 Ready to start TLS"
"DEBUG" 8864 "2021-04-01 09:21:14.070" "Performing SSL/TLS handshake for session 7. Verify certificate: False"
"TCPIP" 8864 "2021-04-01 09:21:14.195" "TCPConnection - TLS/SSL handshake completed. Session Id: 7, Remote IP: 40.92.89.36, Version: TLSv1.2, Cipher: ECDHE-RSA-AES128-GCM-SHA256, Bits: 128"
"SMTPD" 8864 7 "2021-04-01 09:21:14.211" "40.92.89.36" "RECEIVED: EHLO EUR05-DB8-obe.outbound.protection.outlook.com"
"DEBUG" 8864 "2021-04-01 09:21:14.211" "Executing event OnHELO"
"DEBUG" 8864 "2021-04-01 09:21:14.304" "Event completed"

"SMTPD" 8864 7 "2021-04-01 09:21:14.304" "40.92.89.36" "SENT: 250-mail.server.com[nl]250-SIZE 51200000[nl]250 HELP"
"SMTPD" 7808 7 "2021-04-01 09:21:14.320" "40.92.89.36" "RECEIVED: MAIL FROM:<*@outlook.com> SIZE=12374"
"DEBUG" 7808 "2021-04-01 09:21:14.461" "Spam test: SpamTestHeloHost, Score: 1"
"DEBUG" 7808 "2021-04-01 09:21:14.476" "Spam test: SpamTestMXRecords, Score: 0"
"DEBUG" 7808 "2021-04-01 09:21:14.492" "Spam test: SpamTestSPF, Result: Pass"
"DEBUG" 7808 "2021-04-01 09:21:14.492" "Spam test: SpamTestSPF, Score: 0"
"DEBUG" 7808 "2021-04-01 09:21:14.492" "Total spam score: 1"
"SMTPD" 7808 7 "2021-04-01 09:21:14.492" "40.92.89.36" "SENT: 250 OK"
"SMTPD" 4592 7 "2021-04-01 09:21:14.508" "40.92.89.36" "RECEIVED: RCPT TO:<*@server.com>"
"SMTPD" 4592 7 "2021-04-01 09:21:14.523" "40.92.89.36" "SENT: 250 OK"
"SMTPD" 8864 7 "2021-04-01 09:21:14.539" "40.92.89.36" "RECEIVED: DATA"
"DEBUG" 8864 "2021-04-01 09:21:14.539" "Executing event OnSMTPData"
"DEBUG" 8864 "2021-04-01 09:21:14.570" "Event completed"
"SMTPD" 8864 7 "2021-04-01 09:21:14.570" "40.92.89.36" "SENT: 354 OK, send."
spfverify.exe ipaddress host

spfverify.exe 40.92.89.36 outlook.com

Code: Select all

IP address matches SPF record(s) ip range for domain (ExitCode 0)
You need a hmailserver build with Sub OnHELO(oClient) support though

Mail send from Office 365 users always is received from one of outlook servers, you could match the received header with a simple regex

Code: Select all

^([a-z]{3}[\d]{2}\-[a-z]{2}[\d]\-)(obe\.outbound\.protection\.outlook\.com)$

Code: Select all

Dim oRegEx
Set oRegEx = CreateObject("VBScript.RegExp")
oRegEx.IgnoreCase = True
oRegEx.Global = False
oRegEx.Pattern= "^([a-z]{3}[\d]{2}\-[a-z]{2}[\d]\-)(obe\.outbound\.protection\.outlook\.com)$"
If oRegEx.Test(oClient.HELO) Then 
	Call AddGreyList(oClient.IPAddress, oClient.HELO)
	Result.Value = 0
	Exit Sub
End If
Set oRegEx = Nothing

Sub AddGreyList(ByVal strIP, ByVal strHELO)
	dim iReturn : iReturn = 2
	dim hostname : hostname = getDomainName(strHELO) 
	Dim oApp
	Set oApp = CreateObject("hMailServer.Application")
	Call oApp.Authenticate("Administrator", sAdminPassword)
	With LockFile("C:\Program Files (x86)\hMailServer\Temp\greylistwhite.lck")
		On Error Resume Next
		oApp.Settings.AntiSpam.GreyListingWhiteAddresses.Refresh
		If oApp.Settings.AntiSpam.GreyListingWhiteAddresses.ItemByName(strIP) Is Nothing Then
			With CreateObject("WScript.Shell")
				iReturn = .Run("""C:\Program Files (x86)\hMailServer\Events\spfverify.exe"" " & strIP & " " & hostname & "", 0, True)	
			End With
			if iReturn = 0 Then
				EventLog.Write("spfverify.exe " & strIP & " passed for: " & hostname)
				With oApp.Settings.AntiSpam.GreyListingWhiteAddresses.Add
					.Description = Date & " Auto-Added '" & strHELO & "'"
					.IPAddress = strIP
					.Save
				End With
			ElseIf iReturn = 1 Then 		
				EventLog.Write("spfverify.exe " & strIP & " failed for: " & hostname)
			Else		
				EventLog.Write("spfverify.exe command error, spfverify.exe " & strIP & " failed for: " & hostname)
			End if			
		Else
			With oApp.Settings.AntiSpam.GreyListingWhiteAddresses.ItemByName(strIP)
				.Description = Date & " Auto-Added '" & strHELO & "'"
				.Save
			End With
		End If
		oApp.Settings.AntiSpam.GreyListingWhiteAddresses.Refresh
		On Error Goto 0
		.Close '// Close LockFile
	End With 
	Set oApp = Nothing
End Sub

Function getDomainName(byVal strHELO)
	dim aryDomain, str2ndLevel, strTopLevel
	getDomainName = Null
	If Len(strHELO) > 0 Then  	
		aryDomain = Split(strHELO,".")
		If uBound(aryDomain) >= 1 Then
			str2ndLevel = aryDomain(uBound(aryDomain)-1)
			strTopLevel = aryDomain(uBound(aryDomain))			
			getDomainName = str2ndLevel & "." & strTopLevel
		End If
	End If
End Function

Function LockFile(strPath)
	Const Append = 8
	Const Unicode = -1
	With CreateObject("Scripting.FileSystemObject")
		Dim oFile, i
		For i = 0 To 30
			On Error Resume Next
			Set oFile = .OpenTextFile(strPath, Append, True, Unicode)
			If (Not Err.Number = 70) Then
				Set LockFile = oFile
				On Error Goto 0
				Exit For
			End If
			On Error Goto 0
			Wait(1)
		Next
	End With
	Set oFile = Nothing
	If (Err.Number = 70) Then
		EventLog.Write("ERROR: EventHandlers.vbs")
		EventLog.Write("File " & strPath & " is locked and timeout was exceeded.")
		Err.Clear
	ElseIf (Err.Number <> 0) Then
		EventLog.Write("ERROR: EventHandlers.vbs : Function LockFile")
		EventLog.Write("Error       : " & Err.Number)
		EventLog.Write("Error (hex) : 0x" & Hex(Err.Number))
		EventLog.Write("Source      : " & Err.Source)
		EventLog.Write("Description : " & Err.Description)
		Err.Clear
	End If
End Function

Function Wait(sec)
	With CreateObject("WScript.Shell")
		.Run "timeout /NOBREAK /T " & Int(sec), 0, True
		' REM .Run "sleep -m " & Int(sec * 1000), 0, True
		' REM .Run "powershell Start-Sleep -Milliseconds " & Int(sec * 1000), 0, True
	End With
End Function
More info/download of spfverify.exe


GreyListingWhiteAddressesCleanup.vbs (run from task scheduler to remove whiltelistedgreylisting entries after x days)

Code: Select all

'Force error on undeclared variables
Option Explicit
'--------------------------------------------------------------------------
' Cleanup of GreyListingWhiteAddresses 
'--------------------------------------------------------------------------

Dim oApp
Dim oGreyListingWhiteAddresses
Dim oGreyListingWhiteAddress
Dim Days : Days = 90
Dim WLDate
Dim oEventLog

Set oApp = CreateObject("hMailServer.Application") 
Call oApp.Authenticate("Administrator",PASSWORD) ' set your administrator password in this line 
Set oEventlog = CreateObject("hMailServer.EventLog")
Set oGreyListingWhiteAddresses = oApp.Settings.AntiSpam.GreyListingWhiteAddresses
With LockFile("C:\Program Files (x86)\hMailServer\Temp\greylistwhite.lck")
	On Error Resume Next
	For oGreyListingWhiteAddress = oGreyListingWhiteAddresses.Count-1 to 0 Step -1
		If (InStr(1, oGreyListingWhiteAddresses(oGreyListingWhiteAddress).Description, "Auto-Added", 1) > 0) Then
			WLDate = Mid(oGreyListingWhiteAddresses(oGreyListingWhiteAddress).Description,1,(inStr(1, oGreyListingWhiteAddresses(oGreyListingWhiteAddress).Description, " " , 1)-1)) 
			If DateDiff("d", WLDate, Date) > Days then
				oEventLog.Write("GreyListing WhiteAddress Delete: " & oGreyListingWhiteAddresses(oGreyListingWhiteAddress).IPAddress & " - " & oGreyListingWhiteAddresses(oGreyListingWhiteAddress).Description)
				oGreyListingWhiteAddresses(oGreyListingWhiteAddress).Delete
			End If
		End If 
	Next
	On Error Goto 0
	.Close  '// Close LockFile
End With
oGreyListingWhiteAddresses.Refresh
Set oGreyListingWhiteAddresses = nothing
Set oEventlog = nothing
set oApp = nothing

'--------------------------------------------------------

Function LockFile(strPath)
	Const Append = 8
	Const Unicode = -1
	With CreateObject("Scripting.FileSystemObject")
		Dim oFile, i
		For i = 0 To 30
			On Error Resume Next
			Set oFile = .OpenTextFile(strPath, Append, True, Unicode)
			If (Not Err.Number = 70) Then
				Set LockFile = oFile
				On Error Goto 0
				Exit For
			End If
			On Error Goto 0
			Wait(1)
			Set oFile = Nothing
		Next
	End With
	If (Err.Number = 70) Then
		EventLog.Write("ERROR: EventHandlers.vbs")
		EventLog.Write("File " & strPath & " is locked and timeout was exceeded.")
		Err.Clear
	ElseIf (Err.Number <> 0) Then
		EventLog.Write("ERROR: EventHandlers.vbs : Function LockFile")
		EventLog.Write("Error       : " & Err.Number)
		EventLog.Write("Error (hex) : 0x" & Hex(Err.Number))
		EventLog.Write("Source      : " & Err.Source)
		EventLog.Write("Description : " & Err.Description)
		Err.Clear
	End If
End Function

Function Wait(sec)
	With CreateObject("WScript.Shell")
		.Run "timeout /NOBREAK /T " & Int(sec), 0, True
		' REM .Run "sleep -m " & Int(sec * 1000), 0, True
		' REM .Run "powershell Start-Sleep -Milliseconds " & Int(sec * 1000), 0, True
	End With
End Function
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
SorenR
Senior user
Senior user
Posts: 4594
Joined: 2006-08-21 15:38
Location: Denmark

Re: Sub OnHELO(oClient) progress?

Post by SorenR » 2021-04-03 14:12

This works with the HELO/EHLO greeting BUT... It COULD be modified to work with the resolved IP Address thus it can be run on servers pre-OnHELO.

viewtopic.php?p=205049#p205049

Code: Select all

    With CreateObject("DNSLibrary.DNSResolver")
        strLookup = .PTR(strIP)
    End With
SørenR.

Algorithm (noun.)
Word used by programmers when they do not want to explain what they did.

User avatar
RvdH
Senior user
Senior user
Posts: 1466
Joined: 2008-06-27 14:42
Location: Netherlands

Re: Sub OnHELO(oClient) progress?

Post by RvdH » 2021-04-03 15:11

SorenR wrote:
2021-04-03 14:12
This works with the HELO/EHLO greeting BUT... It COULD be modified to work with the resolved IP Address thus it can be run on servers pre-OnHELO.

viewtopic.php?p=205049#p205049
Yes, and that basically is what the OP is asking for
Mine above, is a enhanced version of yours, as it verifies the IP address the mail comes from is defined as being allowed to send from that domain using the SPF-record published by the domain

I have not experienced any issues with 'big ones', like outlook, gmail, twitter, facebook, linkedin....
The only one that seems to fail from time to time is mail arriving from Apple. But that is simply because their SPF record is incomplete and missing some IP's they are sending from, example:

HELO/EHLO: rn2-msbadger06105.apple.com
IP: 17.179.250.100

spfverify.exe 17.179.250.100 apple.com

Code: Select all

IP address does not match SPF record(s) ip range for domain (ExitCode 1)
SorenR wrote:
2021-04-03 14:12

Code: Select all

    With CreateObject("DNSLibrary.DNSResolver")
        strLookup = .PTR(strIP)
    End With
I thought i have seen a post you used the above to verify the HELO/EHLO domain....but i might be mistaken, i can't find it anymore


@Kelden,
So you have options as you see, and these are very "nice" solutions and exactly why the HELO/EHLO event trigger is added in the first place...but yeah it's needs some scripting skills and effort
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
SorenR
Senior user
Senior user
Posts: 4594
Joined: 2006-08-21 15:38
Location: Denmark

Re: Sub OnHELO(oClient) progress?

Post by SorenR » 2021-04-03 17:04

RvdH wrote:
2021-04-03 15:11
SorenR wrote:
2021-04-03 14:12

Code: Select all

    With CreateObject("DNSLibrary.DNSResolver")
        strLookup = .PTR(strIP)
    End With
I thought i have seen a post you used the above to verify the HELO/EHLO domain....but i might be mistaken, i can't find it anymore
Nope, I only verify that the HELO/EHLO follow the RFC.

Code: Select all

    '
    '   Validate HELO/EHLO greeting
    '
    Const strFQDN = "^(?=^.{1,254}$)(^(?:(?!\.|-)([a-z0-9\-\*]{1,63}|([a-z0-9\-]{1,62}[a-z0-9]))\.)+(?:[a-z]{2,})$)$"
    Const strIPv4 = "^\[(?:[0-9]{1,3}\.){3}[0-9]{1,3}\]$"
    Const strIPv6 = "^\[(IPv6)((?:[0-9A-Fa-f]{0,4}:){1,7}(?:(?:(>25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)|[0-9A-Fa-f]{1,4}))\]$"
    strRegEx = strFQDN & "|" & strIPv4 & "|" & strIPv6
    If Not Lookup(strRegEx, oClient.HELO) Then
        Result.Message = "5.3.0 [BAD HELO] Your access to this mail system has been rejected due to the sending MTA's poor reputation. If you believe that this failure is in error, please contact the intended recipient via alternate means."
        Result.Value = 2
        If AutoBan(oClient.IPAddress, "BAD HELO " & oClient.HELO, 7, "d") Then EventLogX.Write( LPad("BAD HELO", 15, " ") & vbTab & LPad(oClient.IPAddress, 16, " ") & vbTab & LPad(" ", 3, " ") & vbTab & LPad(" ", 16, " ") & vbTab & oClient.HELO )
        EventLogX.Write( LPad("REJECT", 15, " ") & vbTab & LPad(oClient.IPAddress, 16, " ") & vbTab & Result.Message )
        ' Disconnect(oClient.IPAddress)
        Set EventLogX = Nothing
        Exit Sub
    End If
SørenR.

Algorithm (noun.)
Word used by programmers when they do not want to explain what they did.

User avatar
nitro
Normal user
Normal user
Posts: 43
Joined: 2018-11-08 16:31
Location: Spain

Re: Sub OnHELO(oClient) progress?

Post by nitro » 2021-05-26 09:56

@RvdH
Hello, a few days ago I received an email from Let's Encrypt which attached the following link:
https://letsencrypt.org/docs/dst-root-c ... ber-2021/#

They seem to indicate a supposed incompatibility of your root certificate with the OpenSSL 1.0.x api.

Currently using a Let's Encrypt certificate in its latest build 5.6.8-B2538.30 hMailserver, I wanted to ask if it is possible that I will run into this problem next September. As always, thank you for your work.


Fifteen minutes later:

OK, don't pay attention to me. :oops:

Version 5.6.8 - Build 2534 (2020-12-31)
Issue 352: Upgrade to OpenSSL 1.1.1i.
Production 5.6.8-B2489.22.RvDH W.Server 2016 Datace [2x Intel Xeon E5-2660 8GB RAM]
Staging 5.7-B2490 W.Server 2008 R2 Stand [Intel Pentium 4 4GB RAM]

Post Reply