hMSLog2Ban - Logfile ip blocker

This section contains user-submitted tutorials.
Post Reply
User avatar
Dravion
Senior user
Senior user
Posts: 1339
Joined: 2015-09-26 11:50
Location: Germany
Contact:

hMSLog2Ban - Logfile ip blocker

Post by Dravion » 2019-07-10 08:14

hMSLog2Ban is a utility which detects a running hMailServer.exe and if its 32 or 64-Bit.
I will find the correct hMailServer log file (without touching the COM API) and scans for "SENT: 550 Unknown user" pattern.
if it finds the pattern, it will lookup the attacker ip address (IPv4 only right now) and blocks the ip address automatically.
Scan interval is by default set to 5 secs. but can be changed to other values (requires stop/start to apply the new timing).
The program is compiled statically, so no Visual Studio Redist DLL's are required. Just download, run and press start.

Download
https://github.com/Dravion/hMSLog2Ban/releases


PS: This is a early testing release. There might be bugs

Sources can be found here: (Requires Visual Studio 2019 Community with MFC-Support options installed to build)

https://github.com/Dravion/hMSLog2Ban

Issue Tracker:
https://github.com/Dravion/hMSLog2Ban/issues

PS:
hMSLog2Ban uses the Windows Filtering Platform (WFP) a Windows builtin C/C++ Subsystem (requires Windows Vista or higher)
which doesnt interfere in any shape or form with Windows Firewall. In fact this means, no Rule will be added, modified or removed from
Windows Firewall. It even runs if Windows Firewall was disabled or the Windows Firewall Process was stopped. Dont try to end hMSLog2Ban
by Process explorer or you need to reboot your System. hMSLog2Ban clears all entries on quitting by itself but if it cannot do the cleanup work,
the banned ip addresses will be banned until the Computer where hMSLog2Ban was running needs to be rebooted to clear the IP bans!
Attachments
hMSLog2Ban_Screenshot.png
64-Bit builds of hMailserver

hMailServer-5.6.+ (HCD) https://github.com/hMailServer-ComDevs/hmailserver
hMailServer-5.6.+ (LTS) https://github.com/Dravion/hMailServer/releases

User avatar
nitro
New user
New user
Posts: 25
Joined: 2018-11-08 16:31
Location: Spain

Re: hMSLog2Ban - Logfile ip blocker

Post by nitro » 2019-07-10 10:39

Ho Ho Ho, it's Christmas in July. Thank you for your present. I'm going to play with him.
Production 5.6.8-B2437.20_OpenSSL_1.0.2s W.Server 2016 Datace [2x Intel Xeon E5-2660 8GB RAM]
Staging 5.7.0-B2428-LibreSSL-x64-LTS W.Server 2008 R2 Stand [Intel Pentium 4 4GB RAM]

User avatar
bagu
Normal user
Normal user
Posts: 187
Joined: 2005-06-17 03:08
Location: France
Contact:

Re: hMSLog2Ban - Logfile ip blocker

Post by bagu » 2019-07-10 12:01

I always wondered if there was not a risk big enough to block a little too largely with this type of system.
But if it is not the case, it is a great system that you propose us there. :wink:
hMailServer 5.6.8 With SpamAssassin 3.4.1

palinka
Senior user
Senior user
Posts: 841
Joined: 2017-09-12 17:57

Re: hMSLog2Ban - Logfile ip blocker

Post by palinka » 2019-07-10 12:36

This looks great!

A couple of questions. I haven't downloaded it yet to try.

1) how do you remove IPs?
2) What if an account simply has a typo? For example, if someone at gmail mis-types the address (with the correct domain, obviously). Its going to block gmail.
3) Is there a filter for the situation in #2?

Always thinking about false positives. :mrgreen:

User avatar
RvdH
Senior user
Senior user
Posts: 746
Joined: 2008-06-27 14:42
Location: Netherlands

Re: hMSLog2Ban - Logfile ip blocker

Post by RvdH » 2019-07-10 13:41

Like it, but where does it store it's blocked ip addresses if it is not using windows Firewall? hMailserver's auto-ban entries?
What if we made a mistake and want to undo a added entry?

Option to specify custom error message to filter on, example: 550 Delivery is not allowed to this address

FYI, it takes my hmailserver_events.log as Current Logfile? Is that right?
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
Dravion
Senior user
Senior user
Posts: 1339
Joined: 2015-09-26 11:50
Location: Germany
Contact:

Re: hMSLog2Ban - Logfile ip blocker

Post by Dravion » 2019-07-10 14:10

RvdH wrote:
2019-07-10 13:41
Like it, but where does it store it's blocked ip addresses if it is not using windows Firewall? hMailserver's auto-bam entries?
It uses the Windows Filtering Platform (WFP).
It was added in Windows Vista and is more Lowlevel than
the Normal Windows Firewall. I believe since Vista, the Entire Windows Firewall sits on top of it. The core Service is the Windows Baseline Filter Engine Windows Service which cannot be stopped, paused or disabled. I was trying to kill it with Sysinternals Process Explorer but it resulted in a Bluescreen ;-)
What if we made a mistake and want to undo a added entry?
For now, just press the Stop or Quit Button and all Bans are gone
Option to specify custom error message to filter on, example: 550 Delivery is not allowed to this address
There is no way to archieve something like this.
It uses only the "current" hMailServer log file for blocking ips after the connection was closed by hMailServer.
FYI, it takes my hmailserver_events.log as Current Logfile? Is that right?
It takes hMailServers current, normal logfile as input.
It requires that logging is enabled plus SMTP logging is enabled in hMailAdmin as well.
64-Bit builds of hMailserver

hMailServer-5.6.+ (HCD) https://github.com/hMailServer-ComDevs/hmailserver
hMailServer-5.6.+ (LTS) https://github.com/Dravion/hMailServer/releases

User avatar
RvdH
Senior user
Senior user
Posts: 746
Joined: 2008-06-27 14:42
Location: Netherlands

Re: hMSLog2Ban - Logfile ip blocker

Post by RvdH » 2019-07-10 14:19

Dravion wrote:
2019-07-10 14:10
It takes hMailServers current, normal logfile as input.
It requires that logging is enabled plus SMTP logging is enabled in hMailAdmin as well.
No, it doesn't

Image
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
RvdH
Senior user
Senior user
Posts: 746
Joined: 2008-06-27 14:42
Location: Netherlands

Re: hMSLog2Ban - Logfile ip blocker

Post by RvdH » 2019-07-10 14:25

Dravion wrote:
2019-07-10 14:10
Option to specify custom error message to filter on, example: 550 Delivery is not allowed to this address
There is no way to archieve something like this.
It uses only the "current" hMailServer log file for blocking ips after the connection was closed by hMailServer.
Why not? In hMailServer we can define error messages ourselves, always starting with 554 (Result.Value = 2) or use the default 'Rejected' (Result.Value = 1)
Would be to difficult to make it filter on any 5xx error code, would it?


FYI: In English it's Sec instead of Sek ;) :lol:
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

palinka
Senior user
Senior user
Posts: 841
Joined: 2017-09-12 17:57

Re: hMSLog2Ban - Logfile ip blocker

Post by palinka » 2019-07-10 14:28

Dravion wrote:
2019-07-10 14:10
What if we made a mistake and want to undo a added entry?
For now, just press the Stop or Quit Button and all Bans are gone
Can you make it load IPs from a database or text file so you don't have to start from scratch everytime you quit or restart?

This could be way more efficient, simpler, faster than firewall banning. I would love to be able to try it with my firewall ban.

User avatar
Dravion
Senior user
Senior user
Posts: 1339
Joined: 2015-09-26 11:50
Location: Germany
Contact:

Re: hMSLog2Ban - Logfile ip blocker

Post by Dravion » 2019-07-10 15:31

RvdH wrote:
2019-07-10 14:19
Dravion wrote:
2019-07-10 14:10
It takes hMailServers current, normal logfile as input.
It requires that logging is enabled plus SMTP logging is enabled in hMailAdmin as well.
No, it doesn't

Image
It seems you didnt click the START Buitton!
You need to click the Start Button to make it work.
You can see if it works in the bottom area

I tested it against official hMailServer 5.6.7 - Build 2425 and my own LTS 5.7.0 64-Bit Releases and it works for me.
Right now its a testing release and not for production, maybe there is a bug in the log parser.

Can you attach your Logfile so i can compare it to hMailServer official?

Take a look at this Screenshot:
inaction.jpg
Last edited by Dravion on 2019-07-10 15:49, edited 1 time in total.
64-Bit builds of hMailserver

hMailServer-5.6.+ (HCD) https://github.com/hMailServer-ComDevs/hmailserver
hMailServer-5.6.+ (LTS) https://github.com/Dravion/hMailServer/releases

User avatar
Dravion
Senior user
Senior user
Posts: 1339
Joined: 2015-09-26 11:50
Location: Germany
Contact:

Re: hMSLog2Ban - Logfile ip blocker

Post by Dravion » 2019-07-10 15:35

RvdH wrote:
2019-07-10 14:25
Dravion wrote:
2019-07-10 14:10
Option to specify custom error message to filter on, example: 550 Delivery is not allowed to this address
There is no way to archieve something like this.
It uses only the "current" hMailServer log file for blocking ips after the connection was closed by hMailServer.
Why not? In hMailServer we can define error messages ourselves, always starting with 554 (Result.Value = 2) or use the default 'Rejected' (Result.Value = 1)
Would be to difficult to make it filter on any 5xx error code, would it?
That is not the point.
hMSLog2Ban is a independent Process and not some sort of VB-Script or COM API attached Client which interacts with hMailServer.
It analyzes hMailServers log file periodically. It cannot write into hMailServers client/Server Socket connections. If you want something like this,
go with the COM API.
64-Bit builds of hMailserver

hMailServer-5.6.+ (HCD) https://github.com/hMailServer-ComDevs/hmailserver
hMailServer-5.6.+ (LTS) https://github.com/Dravion/hMailServer/releases

User avatar
Dravion
Senior user
Senior user
Posts: 1339
Joined: 2015-09-26 11:50
Location: Germany
Contact:

Re: hMSLog2Ban - Logfile ip blocker

Post by Dravion » 2019-07-10 15:44

palinka wrote:
2019-07-10 14:28
Dravion wrote:
2019-07-10 14:10
What if we made a mistake and want to undo a added entry?
For now, just press the Stop or Quit Button and all Bans are gone
Can you make it load IPs from a database or text file so you don't have to start from scratch every time you quit or restart?
This could be way more efficient, simpler, faster than firewall banning. I would love to be able to try it with my firewall ban.
I don't know if this make any sense.
The current log file changes every 24 hours and starts fresh again with an empty log by hMailServer.
hMSLog2Ban can hold up all blocked IP's as long as it runs. If you wish to unblock, just press the Stop Button or quit the App.
Its also not stress out the CPU because the current log contains only entries for max. 24 hours.

IMHO if you want a persistent ip blocking solution, your own project seems to fit way better for this kind of scenario.
64-Bit builds of hMailserver

hMailServer-5.6.+ (HCD) https://github.com/hMailServer-ComDevs/hmailserver
hMailServer-5.6.+ (LTS) https://github.com/Dravion/hMailServer/releases

palinka
Senior user
Senior user
Posts: 841
Joined: 2017-09-12 17:57

Re: hMSLog2Ban - Logfile ip blocker

Post by palinka » 2019-07-10 15:53

Dravion wrote:
2019-07-10 15:44
palinka wrote:
2019-07-10 14:28
Dravion wrote:
2019-07-10 14:10


For now, just press the Stop or Quit Button and all Bans are gone
Can you make it load IPs from a database or text file so you don't have to start from scratch every time you quit or restart?
This could be way more efficient, simpler, faster than firewall banning. I would love to be able to try it with my firewall ban.
I don't know if this make any sense.
The current log file changes every 24 hours and starts fresh again with an empty log by hMailServer.
hMSLog2Ban can hold up all blocked IP's as long as it runs. If you wish to unblock, just press the Stop Button or quit the App.
Its also not stress out the CPU because the current log contains only entries for max. 24 hours.

IMHO if you want a persistent ip blocking solution, your own project seems to fit way better for this kind of scenario.
OK. Thanks for the info. My project is database driven, so it doesn't really matter what is doing the actual blocking. It could be your app, windows firewall, a firewall appliance or router, or anything else that could do the job. I thought your app might be faster & easier to setup than powershell adding and deleting entries from windows firewall.

User avatar
RvdH
Senior user
Senior user
Posts: 746
Joined: 2008-06-27 14:42
Location: Netherlands

Re: hMSLog2Ban - Logfile ip blocker

Post by RvdH » 2019-07-10 15:53

Dravion wrote:
2019-07-10 15:35
RvdH wrote:
2019-07-10 14:25
Dravion wrote:
2019-07-10 14:10

There is no way to archieve something like this.
It uses only the "current" hMailServer log file for blocking ips after the connection was closed by hMailServer.
Why not? In hMailServer we can define error messages ourselves, always starting with 554 (Result.Value = 2) or use the default 'Rejected' (Result.Value = 1)
Would be to difficult to make it filter on any 5xx error code, would it?
That is not the point.
hMSLog2Ban is a independent Process and not some sort of VB-Script or COM API attached Client which interacts with hMailServer.
It analyzes hMailServers log file periodically. It cannot write into hMailServers client/Server Socket connections. If you want something like this,
go with the COM API.
What? You are missing my point completely i think...you currently check for 4 possible reject messages only. hMailServer has plenty more build-in, like the "550 Delivery is not allowed to this address." message (Relaying)
Why limit it to the hardcoded 4 possible reject messages?
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
RvdH
Senior user
Senior user
Posts: 746
Joined: 2008-06-27 14:42
Location: Netherlands

Re: hMSLog2Ban - Logfile ip blocker

Post by RvdH » 2019-07-10 15:58

Dravion wrote:
2019-07-10 15:31
It seems you didnt click the START Buitton!
You need to click the Start Button to make it work.
You can see if it works in the bottom area

Take a look at this Screenshot:
inaction.jpg

Pressing start makes no difference, Different system...same problem
Untitled.png

[EDIT]
Once i delete hmailserver_events.log it picks the right log file....something is wrong with your used logic to open current (active) logfile (do you look at the files creation/modification date for this possibly?)
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
Dravion
Senior user
Senior user
Posts: 1339
Joined: 2015-09-26 11:50
Location: Germany
Contact:

Re: hMSLog2Ban - Logfile ip blocker

Post by Dravion » 2019-07-10 16:13

RvdH wrote:
2019-07-10 15:58
Dravion wrote:
2019-07-10 15:31
It seems you didnt click the START Buitton!
You need to click the Start Button to make it work.
You can see if it works in the bottom area

Take a look at this Screenshot:
inaction.jpg

Pressing start makes no difference, Different system...same problem

Untitled.png

[EDIT]

once i delete hmailserver_events.log it picks the right log file....something is wrong with your used logic to open current (active) logfile

The logic is simple. It iterates trough all *.log files within hMailServer log folder (log path is taken from hMailServer.ini log folder settings).
If it finds a file which is cannot be opened for reading in normal C File *fp open mode (which causes an access error, it is assumed that this
file is the current log file. Maybe this is not the best way to do it.

ps: Do you create additional logfiles in your custom bulld which are locked by hMailServer.exe the the same time in the log folder?

However, i will take a look into it.
Last edited by Dravion on 2019-07-10 16:22, edited 1 time in total.
64-Bit builds of hMailserver

hMailServer-5.6.+ (HCD) https://github.com/hMailServer-ComDevs/hmailserver
hMailServer-5.6.+ (LTS) https://github.com/Dravion/hMailServer/releases

User avatar
RvdH
Senior user
Senior user
Posts: 746
Joined: 2008-06-27 14:42
Location: Netherlands

Re: hMSLog2Ban - Logfile ip blocker

Post by RvdH » 2019-07-10 16:15

Code: Select all

	do
	{
		if (ffd.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) {
			// dont list directory dots			
		}
		else {
			if (!wcscmp(ffd.cFileName, L"hmailserver_awstats.log") == 0) {
				int x = 1;
				HANDLE hFile;
				hFile = CreateFile(ffd.cFileName,
					GENERIC_READ,
					0,
					NULL,
					OPEN_EXISTING,
					FILE_ATTRIBUTE_NORMAL, NULL);

				if (hFile == INVALID_HANDLE_VALUE) {
					found = ffd.cFileName; 					
				}
				CloseHandle(hFile);
			}
		}
	} while (FindNextFile(hFind, &ffd) != 0);
Exclude hmailserver_events.log like you do for hmailserver_awstats.log and any possible ERROR_* logs
...their format (columns) is different anyway

No, i make no additional logfiles in my builds
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
Dravion
Senior user
Senior user
Posts: 1339
Joined: 2015-09-26 11:50
Location: Germany
Contact:

Re: hMSLog2Ban - Logfile ip blocker

Post by Dravion » 2019-07-10 16:19

RvdH wrote:
2019-07-10 16:15

Code: Select all

	do
	{
		if (ffd.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) {
			// dont list directory dots			
		}
		else {
			if (!wcscmp(ffd.cFileName, L"hmailserver_awstats.log") == 0) {
				int x = 1;
				HANDLE hFile;
				hFile = CreateFile(ffd.cFileName,
					GENERIC_READ,
					0,
					NULL,
					OPEN_EXISTING,
					FILE_ATTRIBUTE_NORMAL, NULL);

				if (hFile == INVALID_HANDLE_VALUE) {
					found = ffd.cFileName; 					
				}
				CloseHandle(hFile);
			}
		}
	} while (FindNextFile(hFind, &ffd) != 0);
Exclude hmailserver_events.log like you do for hmailserver_awstats.log and any possible ERROR_* logs
...their format (columns) is different anyway

No, i make no additional logfiles in my builds
Will do.
64-Bit builds of hMailserver

hMailServer-5.6.+ (HCD) https://github.com/hMailServer-ComDevs/hmailserver
hMailServer-5.6.+ (LTS) https://github.com/Dravion/hMailServer/releases

User avatar
katip
Senior user
Senior user
Posts: 642
Joined: 2006-12-22 07:58
Location: Istanbul

Re: hMSLog2Ban - Logfile ip blocker

Post by katip » 2019-07-10 16:26

Dravion wrote:
2019-07-10 08:14
hMSLog2Ban is a utility which detects a running hMailServer.exe and if its 32 or 64-Bit.
I will find the correct hMailServer log file (without touching the COM API) and scans for "SENT: 550 Unknown user" pattern.
thanks for the utility.
last night i finished a VBS which parses current hmailserver_today.log (not awstats!!) SMTPD lines where Instr SENT: 550 or 504 or 535 is true and reads IP in that line.
10+ times occured IPs are banned in IP ranges. if i have time i do some makeup + comments on code and post here.

but what i want to say... i'd suggest you consider looking up "SENT: 504" lines (auth on port 25 disabled) too.
Katip
--
HMS 5.7.0-B2428-LTS-64-bit, MySQL 5.7.24, SA 3.4.2, ClamAV 0.101.2 + SaneS

User avatar
Dravion
Senior user
Senior user
Posts: 1339
Joined: 2015-09-26 11:50
Location: Germany
Contact:

Re: hMSLog2Ban - Logfile ip blocker

Post by Dravion » 2019-07-10 16:45

katip wrote:
2019-07-10 16:26
last night i finished a VBS which parses current hmailserver_today.log (not awstats!!) SMTPD lines where Instr SENT: 550 or 504 or 535 is true and reads IP in that line.
10+ times occured IPs are banned in IP ranges. if i have time i do some makeup + comments on code and post here.
That's awesome!
Maybe i can adapt your script into hMSLog2Ban.
VBScript is nice but it has some limitations. Any testing, especially with various configuration will help to get rid of bugs.
but what i want to say... i'd suggest you consider looking up "SENT: 504" lines (auth on port 25 disabled) too.
Hmm, is this really a Real world issue?
Do you see a lot of SENT: 550 or 504 or 535 log entries?
64-Bit builds of hMailserver

hMailServer-5.6.+ (HCD) https://github.com/hMailServer-ComDevs/hmailserver
hMailServer-5.6.+ (LTS) https://github.com/Dravion/hMailServer/releases

User avatar
RvdH
Senior user
Senior user
Posts: 746
Joined: 2008-06-27 14:42
Location: Netherlands

Re: hMSLog2Ban - Logfile ip blocker

Post by RvdH » 2019-07-10 16:52

Standard hMailServer messages
"SMTPD" 3852 3216 "2019-07-10 15:57:28.792" "102.158.216.36" "SENT: 550 Delivery is not allowed to this address." (not allowing external to external)

"SMTPD" 3532 2077 "2019-07-10 14:37:08.551" "117.86.104.37" "SENT: 504 Authentication not enabled." (DisableAUTHList=25 :!: )

A few of my custom returned error messages:

"SMTPD" 3544 3720 "2019-07-10 16:40:28.290" "108.174.202.220" "SENT: 554 5.7.1 Your access to this mail system has been rejected due to the sending MTA's poor reputation. If you believe that this failure is in error, please contact the intended recipient via alternate means."

"SMTPD" 3544 3728 "2019-07-10 16:40:58.187" "176.9.164.237" "SENT: 554 Rejected - Invalid HELO/EHLO (See RFC2821 4.1.1.1)"

So like katip suggested, for this to be 'really' useful it would be great to be able to define the strings to block on any 500+ statuscodes/strings
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
katip
Senior user
Senior user
Posts: 642
Joined: 2006-12-22 07:58
Location: Istanbul

Re: hMSLog2Ban - Logfile ip blocker

Post by katip » 2019-07-10 19:35

Dravion wrote:
2019-07-10 16:45
Hmm, is this really a Real world issue?
Do you see a lot of SENT: 550 or 504 or 535 log entries?
yes, definitely, from today's log, time 20:00 here:
504 - 4420 hits
550 - 2350 hits

below my script from last night. it's rather a draft, but works perfect. it takes 4-5 secs to finish the job with a 25MB logfile. it runs every 5 mins.
i tried to add comments for each step. as i said, if i have time i do a makeup, avoid some .txt outputs and post it to scripting section.

Code: Select all

'i'm an amateur
'declare rest of vars yourself if you like
Dim oApp : Set oApp = CreateObject("hMailServer.Application")
Call oApp.Authenticate("administrator", "put_your_own_pw_here")
Dim EventLog : Set EventLog = CreateObject("hMailServer.EventLog") 'does nothing yet, use it somewhere below
Dim objFSO : Set objFSO = CreateObject("Scripting.FileSystemObject")
root = "c:\hmailserver" 'assuming "logs" and "temp" folders are under "root"

	'get currently active logfile name
	If month(date) < 10 Then mLog = "0" & month(date) Else mLog = month(date)
	If day(date) < 10 Then dLog = "0" & day(date) Else dLog = day(date)
	logToday = "hmailserver_" & year(date) & "-" & mLog & "-" & dLog & ".log"

'read logfile into an array
	iFile = root & "\logs\" & logToday
	xFile = root & "\temp\ips.txt"
	Set inPutFile = objFSO.OpenTextFile(iFile, 1)
	Set outPutFile = objFSO.OpenTextFile(xFile, 2, True)
	ipContent = inPutFile.ReadAll
	arrIPs = Split(ipContent, VbCrLf)
	inPutFile.Close : Set inPutFile = Nothing

'and lookup & write nasty IPs to ips.txt
	For i=0 to UBound(arrIPs)
		ips = arrIPs(i)		
			If Mid(ips,2,5) = "SMTPD" And (InStr(ips,"SENT: 550") Or InStr(ips,"SENT: 504") Or InStr(ips,"SENT: 535")) Then
				'parse line by tabs to find leading and trailing tabs of IP
				firstTab = InStr(1,ips,Chr(9))
				firstTab = InStr(firstTab + 1,ips,Chr(9))
				firstTab = InStr(firstTab + 1,ips,Chr(9))
				firstTab = InStr(firstTab + 1,ips,Chr(9))
				
				lastTab = InStr(firstTab+1,ips,Chr(9))
				'strip doublequotes and get IP
				ips = Mid(ips,firstTab+2,lastTab-firstTab-3)
				outPutFile.WriteLine(ips)
			End If
	Next
	outPutFile.Close : Set outPutFile = Nothing

'we read IPs into an array
	iFile = root & "\temp\ips.txt"
	xFile = root & "\temp\aban.txt"
	Set inPutFile = objFSO.OpenTextFile(iFile, 1)
	FileIContent = inputfile.ReadAll
	arrFileName = Split(FileIContent, VbCrLf)
	inPutFile.Close : Set inPutFile = Nothing

'and build a dictionary with unique IPs from above array
	Set dicSort = CreateObject("Scripting.Dictionary")
	For i=0 to UBound(arrFileName)
		arrFn = arrfilename(i)
		If arrFn <> "" And Not dicSort.exists(arrFn) Then
			dicSort.Add arrFn, arrFn
		End If
	Next

'and write this unique IPs to aban.txt
	Set outPutFile = objFSO.CreateTextFile(xFile, 2, true)
	dicItems = dicSort.Items
	For i = 0 to dicSort.count - 1
		outPutfile.WriteLine dicItems(i)
	Next
	outPutFile.Close : Set outPutFile = Nothing : Set dicSort = Nothing

'here our 3 musketeers + 1
	iFile = root & "\temp\aban.txt"
	xFile = root & "\temp\abanIP.txt"
	fFile = root & "\temp\ips.txt"
	trigger = 10 'this is a trigger to identify an IP as malicious, YMMV

'read unique IPs from dictionary into an array
	Set inPutFile = objFSO.OpenTextFile(iFile, 1, False, -1)
	'this requires to be in UTF-8 otherwise Split doesn't read it
	'no idea why, hence "-1"
	arrContent = inPutFile.ReadAll
	arrBan = Split(arrContent, VbCrLf)
	inPutFile.Close : Set inPutFile = Nothing

'read full IP set into an array
	Set inPutFile = objFSO.OpenTextFile(fFile, 1)
	arrFContent = inPutFile.ReadAll
	arrFBan = Split(arrFContent, VbCrLf)
	inPutFile.Close : Set inPutFile = Nothing

'check each unique IP with full IP set to see how many times it caused 5xx err
	Set outPutFile = objFSO.OpenTextFile(xFile, 2, True)
	For j = 0 to UBound(arrBan)
		aban = arrBan(j)
		aban = Trim(aban)
		abanIPc = 0
		
		For k = 0 To UBound(arrFBan)
			faban = arrFBan(k)
			faban = Trim(faban)
			If faban = aban Then abanIPc = abanIPc + 1
			If abanIPc > trigger Then Exit For
		Next
	'and write it to abanIP.txt if it has been logged > trigger times
		If abanIPc > trigger Then outPutFile.WriteLine(aban)	
	Next
	outPutFile.Close : Set outPutFile = Nothing

'read "guilty" IPs into an array
	iFile = root & "\temp\abanIP.txt"
	Set inPutFile = objFSO.OpenTextFile(iFile, 1)
	FuncContent = inPutFile.ReadAll
	arrFuncBan = Split(FuncContent, VbCrLf)
	inPutFile.Close : Set inPutFile = Nothing

'and call Function AutoBan() for each of them
	For f = 0 To UBound(arrFuncBan)
		funcBan = arrFuncBan(f)
		funcBan = Trim(funcBan)
		sIPAddress = funcBan
		If sIPAddress <> "" Then Call AutoBan(sIPAddress)
	Next

'cleanup big vars & arrays
'honestly i don't know what VB does with allocated mem after exit, but just in case...
'probably useless
	ipContent = ""
	FileIContent = ""
	arrContent = ""
	arrFContent = ""
	FuncContent = ""
	Erase arrIPs
	Erase arrFileName
	Erase arrBan
	Erase arrFBan
	Erase arrFuncBan

'function stolen from SorenR :)))
Function AutoBan(sIPAddress) : AutoBan = False
   '
   '   sType can be one of the following;
   '   "yyyy" Year, "m" Month, "d" Day, "h" Hour, "n" Minute, "s" Second
   '
   On Error Resume Next
   sReason = "550-504 Ban: "
   iDuration = 1
   sType = "h"
   
   If (oApp.Settings.SecurityRanges.ItemByName(sReason & sIPAddress) Is Nothing) Then
      With oApp.Settings.SecurityRanges.Add
         .Name = sReason & sIPAddress
         .LowerIP = sIPAddress
         .UpperIP = sIPAddress
         .Priority = 20
         .Expires = True
         .ExpiresTime = DateAdd(sType, iDuration, Now())
         .Save
      End With
      AutoBan = True
   End If
   'On Error Goto 0
End Function

Katip
--
HMS 5.7.0-B2428-LTS-64-bit, MySQL 5.7.24, SA 3.4.2, ClamAV 0.101.2 + SaneS

User avatar
SorenR
Senior user
Senior user
Posts: 3142
Joined: 2006-08-21 15:38
Location: Denmark

Re: hMSLog2Ban - Logfile ip blocker

Post by SorenR » 2019-07-10 22:40

Dravion wrote:
2019-07-10 15:44
palinka wrote:
2019-07-10 14:28
Dravion wrote:
2019-07-10 14:10


For now, just press the Stop or Quit Button and all Bans are gone
Can you make it load IPs from a database or text file so you don't have to start from scratch every time you quit or restart?
This could be way more efficient, simpler, faster than firewall banning. I would love to be able to try it with my firewall ban.
I don't know if this make any sense.
The current log file changes every 24 hours and starts fresh again with an empty log by hMailServer.
hMSLog2Ban can hold up all blocked IP's as long as it runs. If you wish to unblock, just press the Stop Button or quit the App.
Its also not stress out the CPU because the current log contains only entries for max. 24 hours.

IMHO if you want a persistent ip blocking solution, your own project seems to fit way better for this kind of scenario.
How to monitor for new logfile or if someone delete the logfile by using WMI... It's part of the this project..
https://www.hmailserver.com/forum/viewt ... 17#p213017

Code: Select all

strQuery = "Select * From __InstanceOperationEvent Within " & intInterval & _
           " Where Targetinstance Isa 'CIM_DataFile' And TargetInstance.Drive='" & strDrive & _
           "' And TargetInstance.Path='" & strFolder & "'"
Set oWMIService = GetObject("winmgmts:{impersonationLevel=impersonate}!\\" & strComputer & "\root\cimv2")
Set colEvents = oWMIService.ExecNotificationQuery(strQuery)

Do
    Set oEvent = colEvents.NextEvent()
    Set oTargetInst = oEvent.TargetInstance
    Select Case oEvent.Path_.Class
        Case "__InstanceDeletionEvent"
            If 0 = StrComp(oTargetInst.Name, strLongFileName, 1) Then
                WScript.echo oTargetInst.Name & " has been deleted. Exiting."
                WScript.quit
            End If
        Case "__InstanceModificationEvent"
            If 0 = StrComp(oTargetInst.Name, strLongFileName, 1) Then
                tailFile(oTargetInst.Name)
            End If
        Case "__InstanceCreationEvent"
            If 0 = StrComp(Left(get_FileName(oTargetInst.Name), 12), Left(get_FileName(strLongFileName), 12), 1) Then
                WScript.echo oTargetInst.Name & " has been created. Loading new logfile."
                strLongFileName = oTargetInst.Name
                intLastRunLineinFile = 0
            End If
    End Select
Loop
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
Dravion
Senior user
Senior user
Posts: 1339
Joined: 2015-09-26 11:50
Location: Germany
Contact:

Re: hMSLog2Ban - Logfile ip blocker

Post by Dravion » 2019-07-11 00:21

Right now hMSLog2Ban iterates trough all *.log files within hMailServers Log folder and is trying to open any log file in exclusive mode until a file was found which cannot be opened because it is allready opened by another process.
If the file is not awstats log, it is assumed it is the current log file. This current log file detection runs before any new scan is done, so it switches automatically to the new current log file if hMailServer.exe switches to a new current log file.

Problem is, it doesnt exclude the event.log and Error logs ect.(as RvdH allready found out) and doesnt detect if the locked file was locked by hMailServer.exe or some other process. This needs to be corrected asap.
64-Bit builds of hMailserver

hMailServer-5.6.+ (HCD) https://github.com/hMailServer-ComDevs/hmailserver
hMailServer-5.6.+ (LTS) https://github.com/Dravion/hMailServer/releases

User avatar
SorenR
Senior user
Senior user
Posts: 3142
Joined: 2006-08-21 15:38
Location: Denmark

Re: hMSLog2Ban - Logfile ip blocker

Post by SorenR » 2019-07-11 02:04

Dravion wrote:
2019-07-11 00:21
Right now hMSLog2Ban iterates trough all *.log files within hMailServers Log folder and is trying to open any log file in exclusive mode until a file was found which cannot be opened because it is allready opened by another process.
If the file is not awstats log, it is assumed it is the current log file. This current log file detection runs before any new scan is done, so it switches automatically to the new current log file if hMailServer.exe switches to a new current log file.

Problem is, it doesnt exclude the event.log and Error logs ect.(as RvdH allready found out) and doesnt detect if the locked file was locked by hMailServer.exe or some other process. This needs to be corrected asap.
Did you ever consider that someone would uncheck "keep files open" ??

Anyways ...

Das "tail -f" für vbs-Projekt, über das ich geschrieben habe, hat mich darüber nachdenken lassen, was ich auf einem ausgelasteten Server tun soll. Wenn Sie alle 5 Sekunden auslösen, müssen Sie möglicherweise 3-500 Textzeilen verarbeiten. Ich dachte daran, einen "Controller" und eine Reihe von "Arbeitern" zu haben, um in Spitzenzeiten einen Rückstand zu vermeiden.

Als ich bei Belle Systems, Digiquant, Intec, CSG mit der VoIP / ISP / ITSP-Abrechnung (IMS / DCP) arbeitete, ließ ich diesen Object Broker auf CORBA aufbauen, der steuern sollte, wohin Aufträge gehen. "Arbeiter" würden sich "anmelden", um die Leistung zu steigern, fügten Sie einfach mehr Arbeiter hinzu. Auch die Arbeiter könnten lokal oder auf anderen Maschinen installiert werden - eine vollständige dezentrale Struktur. Wir hatten ein (!) Abrechnungssystem in 4 Ländern an mehreren Standorten installiert.

Ich habe versucht, dies mit VBScript zu tun, aber Sie können kein Objekt von Skript zu Skript weitergeben.
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
Dravion
Senior user
Senior user
Posts: 1339
Joined: 2015-09-26 11:50
Location: Germany
Contact:

Re: hMSLog2Ban - Logfile ip blocker

Post by Dravion » 2019-07-11 03:34

SorenR wrote:
2019-07-11 02:04
Dravion wrote:
2019-07-11 00:21
Right now hMSLog2Ban iterates trough all *.log files within hMailServers Log folder and is trying to open any log file in exclusive mode until a file was found which cannot be opened because it is allready opened by another process.
If the file is not awstats log, it is assumed it is the current log file. This current log file detection runs before any new scan is done, so it switches automatically to the new current log file if hMailServer.exe switches to a new current log file.

Problem is, it doesnt exclude the event.log and Error logs ect.(as RvdH allready found out) and doesnt detect if the locked file was locked by hMailServer.exe or some other process. This needs to be corrected asap.
Did you ever consider that someone would uncheck "keep files open" ??
Good point. I will check for it as well.
64-Bit builds of hMailserver

hMailServer-5.6.+ (HCD) https://github.com/hMailServer-ComDevs/hmailserver
hMailServer-5.6.+ (LTS) https://github.com/Dravion/hMailServer/releases

User avatar
Dravion
Senior user
Senior user
Posts: 1339
Joined: 2015-09-26 11:50
Location: Germany
Contact:

Re: hMSLog2Ban - Logfile ip blocker

Post by Dravion » 2019-07-14 09:57

Fix:
hmailserver_awstats.log
hmailserver_events.log
hmailserver_backup.log
ERROR_hmailserver*.logs

Are now excluded.
*Source is in sync with the compiled EXE
*Codesigned and Virustotal Hash updated

https://github.com/Dravion/hMSLog2Ban/r ... ag/1.0.0.1
64-Bit builds of hMailserver

hMailServer-5.6.+ (HCD) https://github.com/hMailServer-ComDevs/hmailserver
hMailServer-5.6.+ (LTS) https://github.com/Dravion/hMailServer/releases

User avatar
jimimaseye
Moderator
Moderator
Posts: 7957
Joined: 2011-09-08 17:48

Re: hMSLog2Ban - Logfile ip blocker

Post by jimimaseye » 2019-07-14 11:02

What happens to those of us that have split log files using this setting:

Code: Select all

SepSvcLogs=1

; This key tells hmailserver to split SMTP/IMAP/POP into their own log files such as

hmailserver_SMTP_2010-10-24.log, 
hmailserver_IMAP_2010-10-24.log, 
hmailserver_POP3_2010-10-24.log

; Default is to have all services logged together in 1 file.
I would recommend you scan for this setting and if in use then target only the open hmailserver_SMTP log file.

[Entered by mobile. Excuse my spelling.]
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
Dravion
Senior user
Senior user
Posts: 1339
Joined: 2015-09-26 11:50
Location: Germany
Contact:

Re: hMSLog2Ban - Logfile ip blocker

Post by Dravion » 2019-07-15 00:16

jimimaseye wrote:
2019-07-14 11:02
What happens to those of us that have split log files using this setting:

Code: Select all

SepSvcLogs=1

; This key tells hmailserver to split SMTP/IMAP/POP into their own log files such as

hmailserver_SMTP_2010-10-24.log, 
hmailserver_IMAP_2010-10-24.log, 
hmailserver_POP3_2010-10-24.log

; Default is to have all services logged together in 1 file.
I would recommend you scan for this setting and if in use then target only the open hmailserver_SMTP log file.
Interresting Info!
It would be possible to scan for this switch but i want to block IPs for IMAP and POP3 false login attemps as well.

I also plan to check if the sending SMTP-Client resolves to a MX record or not.If it has no MX record its allmost 100% a Spammer.
64-Bit builds of hMailserver

hMailServer-5.6.+ (HCD) https://github.com/hMailServer-ComDevs/hmailserver
hMailServer-5.6.+ (LTS) https://github.com/Dravion/hMailServer/releases

User avatar
nitro
New user
New user
Posts: 25
Joined: 2018-11-08 16:31
Location: Spain

Re: hMSLog2Ban - Logfile ip blocker

Post by nitro » 2019-07-15 12:03

On my server, RAM consumption is triggered up to stratospheric levels in the service Host: Local service. It exceeds 1GB.
Production 5.6.8-B2437.20_OpenSSL_1.0.2s W.Server 2016 Datace [2x Intel Xeon E5-2660 8GB RAM]
Staging 5.7.0-B2428-LibreSSL-x64-LTS W.Server 2008 R2 Stand [Intel Pentium 4 4GB RAM]

User avatar
Dravion
Senior user
Senior user
Posts: 1339
Joined: 2015-09-26 11:50
Location: Germany
Contact:

Re: hMSLog2Ban - Logfile ip blocker

Post by Dravion » 2019-07-15 14:02

Is this related to hMSLog2Ban?
64-Bit builds of hMailserver

hMailServer-5.6.+ (HCD) https://github.com/hMailServer-ComDevs/hmailserver
hMailServer-5.6.+ (LTS) https://github.com/Dravion/hMailServer/releases

User avatar
nitro
New user
New user
Posts: 25
Joined: 2018-11-08 16:31
Location: Spain

Re: hMSLog2Ban - Logfile ip blocker

Post by nitro » 2019-07-15 15:06

Yes, it is a service that is activated when using hmslog2ban, in Win2016 Server
Production 5.6.8-B2437.20_OpenSSL_1.0.2s W.Server 2016 Datace [2x Intel Xeon E5-2660 8GB RAM]
Staging 5.7.0-B2428-LibreSSL-x64-LTS W.Server 2008 R2 Stand [Intel Pentium 4 4GB RAM]

User avatar
RvdH
Senior user
Senior user
Posts: 746
Joined: 2008-06-27 14:42
Location: Netherlands

Re: hMSLog2Ban - Logfile ip blocker

Post by RvdH » 2019-07-15 15:20

Same on Windows Server 2012, svchost > firewall service & base filtering engine & diagnostic policy server

Innitially it is no problem as it seems, but let it run some hours and memusage goes bazirk
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
Dravion
Senior user
Senior user
Posts: 1339
Joined: 2015-09-26 11:50
Location: Germany
Contact:

Re: hMSLog2Ban - Logfile ip blocker

Post by Dravion » 2019-07-16 08:00

Seems to be a stack deallocation Problem after adding C++ Collections without resetting it correctly before. Will look into it
64-Bit builds of hMailserver

hMailServer-5.6.+ (HCD) https://github.com/hMailServer-ComDevs/hmailserver
hMailServer-5.6.+ (LTS) https://github.com/Dravion/hMailServer/releases

Post Reply