Order of "Greylisting" "White Listing"

Use this forum if you have installed hMailServer and want to ask a question related to a production release of hMailServer. Before posting, please read the troubleshooting guide. A large part of all reported issues are already described in detail here.
Post Reply
twaldorf
New user
New user
Posts: 22
Joined: 2009-07-13 11:29

Order of "Greylisting" "White Listing"

Post by twaldorf » 2019-06-14 08:28

Hello,

I want to understand in which order hMailServer will do the Anti-Spam tests.

We activated Greylisting and want to Whitelist a customer who use Office 365. I added *@domain.tld in "White Listing" but it seems that nevertheless "Greylisting" will checked before "White Listing". Can someone confirm this?

The "White Listing" Tab within Greylisting is useless for this case because Office 365 use thousands of IPs and I don't want to whitelist all of them.

If my suggestion is right: Is there a way to change the order so that no other tests will be done if a domain/address is in "White Listing"?

Thanks and best regards,

Thorsten

User avatar
mattg
Moderator
Moderator
Posts: 20296
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Order of "Greylisting" "White Listing"

Post by mattg » 2019-06-14 08:49

twaldorf wrote:
2019-06-14 08:28
but it seems that nevertheless "Greylisting" will checked before "White Listing". Can someone confirm this?
Yes, that is correct
twaldorf wrote:
2019-06-14 08:28
The "White Listing" Tab within Greylisting is useless for this case because Office 365 use thousands of IPs and I don't want to whitelist all of them.
Correct
twaldorf wrote:
2019-06-14 08:28
If my suggestion is right: Is there a way to change the order so that no other tests will be done if a domain/address is in "White Listing"?
No

You can bypass greylisting on SPF pass though, but that has it's own limitations.
(including that all softfail ~all and refuse to use spf +all records are automatically passed)
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
jimimaseye
Moderator
Moderator
Posts: 8174
Joined: 2011-09-08 17:48

Re: Order of "Greylisting" "White Listing"

Post by jimimaseye » 2019-06-14 09:09

If you are using Spamassassin, you can whitelist the sender/domain in its setup (local.cf) so it will immediately bypass not perform any tests.
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

twaldorf
New user
New user
Posts: 22
Joined: 2009-07-13 11:29

Re: Order of "Greylisting" "White Listing"

Post by twaldorf » 2019-06-14 09:12

Thanks both of you !!!!

Yes, I use SpamAssassin and will give it a try. This solutions seems to be the best option for me.

User avatar
jim.bus
Senior user
Senior user
Posts: 304
Joined: 2011-05-28 11:49
Location: US

Re: Order of "Greylisting" "White Listing"

Post by jim.bus » 2019-06-15 09:04

twaldorf,

See this section copied from the hMailServer Help documentation below. This section is not describing the White listing Tab function. It is an option off the Anti-Spam navigation tree directly under Grey listing in the tree. This is what I use to White list a specific email address or Domain which is what I believe you are trying to do. According to the documentation you can White list an email id or Domain regardless of what IP Address it comes from. Since this White Listing option is a part of the Anti-Spam selection in hMailAdmin this according to the documentation should cause all Spam checking to not be performed and Grey Listing is a part of the Anti-Spam checking.

If you did try this method, did you make sure you specified the range of IP Addresses to be 0.0.0.0 through 255.255.255.255 on the White List record you created.

From hMailServer Help Documentation:

Whitelisting

General

hMailServer includes a number of anti-spam features. In some cases, you want certain senders to bypass all these. For example, a specific IP address may have been blacklisted by mistake, but you still want to be able to receive email originating from this IP address. Another example is that you may expect email from a specific sender, and you don't want to risk to loose this email if it's classified as spam.

To do this, you can add white-list records to the configuration. If hMailServer receives an email from a source matching one of these records, hMailServer will not try to determine whether the email is spam. To add a whitelist record, start hMailServer Administrator, and navigate to Settings, Spam protection, White listing. For every white list record, you can specify a description, an lower and upper IP address and an email address.

Before performing spam protection, hMailServer determines the IP address of the sender. When this has been done, hMailServer goes through the list of white list records. If a record matching the IP address is found, hMailServer checks whether the email address specified in the white list record matches. If so, spam protection is bypassed for this email.

User avatar
RvdH
Senior user
Senior user
Posts: 817
Joined: 2008-06-27 14:42
Location: Netherlands

Re: Order of "Greylisting" "White Listing"

Post by RvdH » 2019-06-15 09:24

Dynamic GreyWhiteListing?
This approach does not only verify the SPF check but it also verifies the received header so it matches the domain it supposed to be arriving from

You need a hmailserver build with Sub OnHELO(oClient) support though

Mail send from Office 365 users always is received from one of outlook servers, you could match the received header with a simple regex

Code: Select all

^([a-z]{3}[\d]{2}\-[a-z]{2}[\d]\-)(obe\.outbound\.protection\.outlook\.com)$

Code: Select all

Dim oRegEx
Set oRegEx = CreateObject("VBScript.RegExp")
oRegEx.IgnoreCase = True
oRegEx.Global = False
oRegEx.Pattern= "^([a-z]{3}[\d]{2}\-[a-z]{2}[\d]\-)(obe\.outbound\.protection\.outlook\.com)$"
If oRegEx.Test(oClient.HELO) Then 
	Call AddGreyList(oClient.IPAddress, oClient.HELO)
	Result.Value = 0
	Exit Sub
End If
Set oRegEx = Nothing

Sub AddGreyList(ByVal strIP, ByVal strHELO)
	dim iReturn : iReturn = 2
	dim hostname : hostname = getDomainName(strHELO) 
	Dim oApp
	Set oApp = CreateObject("hMailServer.Application")
	Call oApp.Authenticate("Administrator", sAdminPassword)
	With LockFile("C:\Program Files (x86)\hMailServer\Temp\greylistwhite.lck")
		On Error Resume Next
		oApp.Settings.AntiSpam.GreyListingWhiteAddresses.Refresh
		If oApp.Settings.AntiSpam.GreyListingWhiteAddresses.ItemByName(strIP) Is Nothing Then
			With CreateObject("WScript.Shell")
				iReturn = .Run("""C:\Program Files (x86)\hMailServer\Events\spfverify.exe"" " & strIP & " " & hostname & "", 0, True)	
			End With
			if iReturn = 0 Then
				EventLog.Write("spfverify.exe " & strIP & " passed for: " & hostname)
				With oApp.Settings.AntiSpam.GreyListingWhiteAddresses.Add
					.Description = Date & " Auto-Added '" & strHELO & "'"
					.IPAddress = strIP
					.Save
				End With
			ElseIf iReturn = 1 Then 		
				EventLog.Write("spfverify.exe " & strIP & " failed for: " & hostname)
			Else		
				EventLog.Write("spfverify.exe command error, spfverify.exe " & strIP & " failed for: " & hostname)
			End if			
		Else
			With oApp.Settings.AntiSpam.GreyListingWhiteAddresses.ItemByName(strIP)
				.Description = Date & " Auto-Added '" & strHELO & "'"
				.Save
			End With
		End If
		oApp.Settings.AntiSpam.GreyListingWhiteAddresses.Refresh
		On Error Goto 0
		.Close '// Close LockFile
	End With 
	Set oApp = Nothing
End Sub

Function getDomainName(byVal strHELO)
	dim aryDomain, str2ndLevel, strTopLevel
	getDomainName = Null
	If Len(strHELO) > 0 Then  	
		aryDomain = Split(strHELO,".")
		If uBound(aryDomain) >= 1 Then
			str2ndLevel = aryDomain(uBound(aryDomain)-1)
			strTopLevel = aryDomain(uBound(aryDomain))			
			getDomainName = str2ndLevel & "." & strTopLevel
		End If
	End If
End Function

Function LockFile(strPath)
	Const Append = 8
	Const Unicode = -1
	With CreateObject("Scripting.FileSystemObject")
		Dim oFile, i
		For i = 0 To 30
			On Error Resume Next
			Set oFile = .OpenTextFile(strPath, Append, True, Unicode)
			If (Not Err.Number = 70) Then
				Set LockFile = oFile
				On Error Goto 0
				Exit For
			End If
			On Error Goto 0
			Wait(1)
		Next
	End With
	Set oFile = Nothing
	If (Err.Number = 70) Then
		EventLog.Write("ERROR: EventHandlers.vbs")
		EventLog.Write("File " & strPath & " is locked and timeout was exceeded.")
		Err.Clear
	ElseIf (Err.Number <> 0) Then
		EventLog.Write("ERROR: EventHandlers.vbs : Function LockFile")
		EventLog.Write("Error       : " & Err.Number)
		EventLog.Write("Error (hex) : 0x" & Hex(Err.Number))
		EventLog.Write("Source      : " & Err.Source)
		EventLog.Write("Description : " & Err.Description)
		Err.Clear
	End If
End Function

Function Wait(sec)
	With CreateObject("WScript.Shell")
		.Run "timeout /NOBREAK /T " & Int(sec), 0, True
		' REM .Run "sleep -m " & Int(sec * 1000), 0, True
		' REM .Run "powershell Start-Sleep -Milliseconds " & Int(sec * 1000), 0, True
	End With
End Function
More info/download of spfverify.exe
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
jimimaseye
Moderator
Moderator
Posts: 8174
Joined: 2011-09-08 17:48

Re: Order of "Greylisting" "White Listing"

Post by jimimaseye » 2019-06-15 10:52

jim.bus wrote:
2019-06-15 09:04
twaldorf,

See this section copied from the hMailServer Help documentation below. This section is not describing the White listing Tab function. It is an option off the Anti-Spam navigation tree directly under Grey listing in the tree. This is what I use to White list a specific email address or Domain which is what I believe you are trying to do. ...
I don't think you understood what he wants.

It seems the OP wants to whitelist a SENDER DOMAIN from spam checking once it has been confirmed to be exempt from greylisting with greylist whitelisting as he said:
If my suggestion is right: Is there a way to change the order so that no other tests will be done if a domain/address is in "White Listing"?
I assume this because he talks about the whitelist he uses requires ip addresses (which the anti-spam one does but the greenish whitelist doesn't) and because we know greylist whitelist does work.

I also assume he has enabled greylist against the domain as well as simply activating the grey list feature (hint).

Spam check whitelisting is only maintained by IP address. He cannot whitelist the ip range because the sender uses 365 (thousands of sending server ranges) and even if he could enter them he would then be whitelisting every other sender that uses 365 too.


IF I AM COMPLETELY WRONG in my assumptions and all he wants to do is whitelist a single user/domain (external.tld) from being subject to greylisting (only) then he shouldn't have a problem by entering it in the greylist whitelist with ip 0.0.0.0 to 255.255.255.255. Of course the email will still be subjected to the other anti-spam measures.

[Entered by mobile. Excuse my spelling.]
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
SorenR
Senior user
Senior user
Posts: 3228
Joined: 2006-08-21 15:38
Location: Denmark

Re: Order of "Greylisting" "White Listing"

Post by SorenR » 2019-06-15 11:52

jimimaseye wrote:
2019-06-15 10:52
jim.bus wrote:
2019-06-15 09:04
twaldorf,

See this section copied from the hMailServer Help documentation below. This section is not describing the White listing Tab function. It is an option off the Anti-Spam navigation tree directly under Grey listing in the tree. This is what I use to White list a specific email address or Domain which is what I believe you are trying to do. ...
I don't think you understood what he wants.

It seems the OP wants to whitelist a SENDER DOMAIN from spam checking once it has been confirmed to be exempt from greylisting with greylist whitelisting as he said:
If my suggestion is right: Is there a way to change the order so that no other tests will be done if a domain/address is in "White Listing"?
I assume this because he talks about the whitelist he uses requires ip addresses (which the anti-spam one does but the greenish whitelist doesn't) and because we know greylist whitelist does work.

I also assume he has enabled greylist against the domain as well as simply activating the grey list feature (hint).

Spam check whitelisting is only maintained by IP address. He cannot whitelist the ip range because the sender uses 365 (thousands of sending server ranges) and even if he could enter them he would then be whitelisting every other sender that uses 365 too.


IF I AM COMPLETELY WRONG in my assumptions and all he wants to do is whitelist a single user/domain (external.tld) from being subject to greylisting (only) then he shouldn't have a problem by entering it in the greylist whitelist with ip 0.0.0.0 to 255.255.255.255. Of course the email will still be subjected to the other anti-spam measures.

[Entered by mobile. Excuse my spelling.]
You can only whitelist a single IP address from Greylisting, which is why I originally made the "Dynamic Greylist" code.

The complexity of the code was eventually the reson why I skipped Greylisting all together in favor of the 20 second delay at all primary triggers - which appears to have the exact same effect as Greylisting with respect to SPAM senders.
SørenR.

“With age comes wisdom, but sometimes age comes alone.”
- Oscar Wilde

User avatar
SorenR
Senior user
Senior user
Posts: 3228
Joined: 2006-08-21 15:38
Location: Denmark

Re: Order of "Greylisting" "White Listing"

Post by SorenR » 2019-06-15 11:59

twaldorf wrote:
2019-06-14 08:28
Hello,

I want to understand in which order hMailServer will do the Anti-Spam tests.

We activated Greylisting and want to Whitelist a customer who use Office 365. I added *@domain.tld in "White Listing" but it seems that nevertheless "Greylisting" will checked before "White Listing". Can someone confirm this?

The "White Listing" Tab within Greylisting is useless for this case because Office 365 use thousands of IPs and I don't want to whitelist all of them.

If my suggestion is right: Is there a way to change the order so that no other tests will be done if a domain/address is in "White Listing"?

Thanks and best regards,

Thorsten
Short answer: NO, Greylisting and Whitelisting are two separate items not related.

I have skipped GreyListing and use this:

Code: Select all

Public Function Wait(sec)
   With CreateObject("WScript.Shell")
      .Run "powershell Start-Sleep -Milliseconds " & Int(sec * 1000), 0, True
   End With
End Function

Sub OnClientConnect(oClient)
   '
   '   Filter out "impatient" servers. Alternative to GreyListing.
   '
   If (oClient.Port = 25) Then Wait(20)
End Sub

Sub OnHELO(oClient)
   '
   '   Filter out "impatient" servers. Alternative to GreyListing.
   '
   If (oClient.Port = 25) Then Wait(20)
End Sub

Sub OnSMTPData(oClient, oMessage)
   '
   '   Filter out "impatient" servers. Alternative to GreyListing.
   '
   If (oClient.Username = "") Then Wait(20)
End Sub
There are reports of problems with Anti-SPAM check for DKIM while using the Wait(20). I expect this to be a bug in the DKIM code. I use SpamAssassin for all SPAM tests ( SPF, HELO, MX and DKIM ) so I have not exerienced the problem myself.

PS. I use port 25 for incoming mail only, clients use a different port. :wink:
SørenR.

“With age comes wisdom, but sometimes age comes alone.”
- Oscar Wilde

User avatar
jimimaseye
Moderator
Moderator
Posts: 8174
Joined: 2011-09-08 17:48

Re: Order of "Greylisting" "White Listing"

Post by jimimaseye » 2019-06-15 16:43

SorenR wrote:
2019-06-15 11:52

You can only whitelist a single IP address from Greylisting, which is why I originally made the "Dynamic Greylist" code.
Oh really, didn't realise (i didn't check).
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

Post Reply