Ban spammers based on prior spamming

This section contains scripts that hMailServer has contributed with. hMailServer 5 is needed to use these.
Post Reply
palinka
Senior user
Senior user
Posts: 1393
Joined: 2017-09-12 17:57

Ban spammers based on prior spamming

Post by palinka » 2019-12-04 22:56

I noticed a pattern for certain types of spammers. I do not believe they are bots. They have a real domain and PTR. I think they just blast out as much as they can for a day or 2 until spamhaus and others pick them up and they start getting blocked all over the place. But for that day or 2 they can get mucho spam out the door.

The other thing is that they use subdomains on other IPs, so they can send from many IPs using the same domain - and all of them have valid PTRs. I've been using Soren's xml solution to filter these guys out using regex for the main domain, but that requires a manual entry after noticing a flood of spam. I wanted to automate it, but I couldn't figure out how to programmatically write to xml. I also tried json but had issues with that as well. Finally, I settled on a database solution because its simpler for me to create.

The script works in 2 parts:

1) If a message is marked as spam, add "domain.tld" (not the full subdomain.domain.tld) to the database.
2) Check HELO against the database - if the domain.tld portion of the HELO matches the database AND there are at least 3 entries for domain.tld (3 bonafide spams), the reject the connection.

I think it probably still needs some fine tuning to protect against false positives, but the concept is sound.

MySQL:

Code: Select all

CREATE TABLE `hm_catchspam` (
  `timestamp` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
  `domain` varchar(25) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
COMMIT;

Eventhandlers.vbs:

Code: Select all

Function CatchSpam(spamDomain)
	Dim strSQL, oDB : Set oDB = GetDatabaseObject
	strSQL = "INSERT INTO hm_catchspam (domain) VALUES (spamDomain);"
	Call oDB.ExecuteSQL(strSQL)
End Function

Function IsCatchSpam(spamDomain) : IsCatchSpam = False
    Dim oRecord, oConn : Set oConn = CreateObject("ADODB.Connection")
    oConn.Open "Driver={MariaDB ODBC 3.0 Driver}; Server=localhost; Database=hmailserver; User=hmailserver; Password=supersecretpassword;"

    If oConn.State <> 1 Then
		EventLog.Write( "Function IsCatchSpam - ERROR: Could not connect to database" )
        Exit Function
    End If

    Set oRecord = oConn.Execute("SELECT COUNT(domain) AS countdomain FROM hm_catchspam WHERE domain = '" & spamDomain & "'")
    Do Until oRecord.EOF
        m_CountDomain = oRecord("countdomain")
        oRecord.MoveNext
    Loop
    oConn.Close
    Set oRecord = Nothing
	If (CInt(m_CountDomain) > 2) Then IsCatchSpam = True
End Function

Sub OnHELO(oClient)

	'	Reject on CatchSpam
	strRegEx = "[a-z0-9]+\.[a-z]{2,12}$"
	Set Matches = oLookup(strRegEx, oClient.HELO, False)
	For Each Match In Matches
	   spamDomain = Match.Value
	Next
	If IsCatchSpam(spamDomain) Then
		Result.Value = 2
		Result.Message = ". 19 Your access to this mail system has been rejected due to the sending MTA's poor reputation. If you believe that this failure is in error, please contact the intended recipient via alternate means."
		Call AutoBan(oClient.IPAddress, "CatchSpam - " & oClient.IpAddress, 1, "h") ' https://www.hmailserver.com/forum/viewtopic.php?p=186868#p186868
		Exit Sub
	End If	

End Sub

Sub OnAcceptMessage(oClient, oMessage)

	'	Record entries for CatchSpam
	If (oMessage.HeaderValue("X-hMailServer-Spam") = "YES") Then 
		strRegEx = "[a-z0-9]+\.[a-z]{2,12}$"
		Set Matches = oLookup(strRegEx, oClient_HELO, False)
		For Each Match In Matches
		   spamDomain = Match.Value
		Next
		Call CatchSpam(spamDomain)
	End If

End Sub

palinka
Senior user
Senior user
Posts: 1393
Joined: 2017-09-12 17:57

Re: Ban spammers based on prior spamming

Post by palinka » 2019-12-05 03:27

oops.... some remnants from testing it. Here is adjusted script:

EventHandlers.vbs:

Code: Select all

Function CatchSpam(spamDomain)
	Dim strSQL, oDB : Set oDB = GetDatabaseObject
	strSQL = "INSERT INTO hm_catchspam (domain) VALUES ('" & spamDomain & "');"
	Call oDB.ExecuteSQL(strSQL)
End Function

Function IsCatchSpam(spamDomain) : IsCatchSpam = False
	Dim m_CountDomain
    Dim oRecord, oConn : Set oConn = CreateObject("ADODB.Connection")
    oConn.Open "Driver={MariaDB ODBC 3.0 Driver}; Server=localhost; Database=hmailserver; User=hmailserver; Password=supersecretpassword;"

    If oConn.State <> 1 Then
		EventLog.Write( "Function IsCatchSpam - ERROR: Could not connect to database" )
        Exit Function
    End If

    Set oRecord = oConn.Execute("SELECT COUNT(domain) AS countdomain FROM hm_catchspam WHERE domain = '" & spamDomain & "'")
    Do Until oRecord.EOF
        m_CountDomain = oRecord("countdomain")
        oRecord.MoveNext
    Loop
    oConn.Close
    Set oRecord = Nothing
	If (CInt(m_CountDomain) > 2) Then IsCatchSpam = True
End Function

Sub OnHELO(oClient)

	'	Reject on CatchSpam
	Dim spamDomain
	strRegEx = "[a-z0-9]+\.[a-z]{2,12}$"
	Set Matches = oLookup(strRegEx, oClient.HELO, False)
	For Each Match In Matches
	   spamDomain = Match.Value
	Next
	If IsCatchSpam(spamDomain) Then
		Result.Value = 2
		Result.Message = ". 19 Your access to this mail system has been rejected due to the sending MTA's poor reputation. If you believe that this failure is in error, please contact the intended recipient via alternate means."
		Call AutoBan(oClient.IPAddress, "CatchSpam - " & oClient.IpAddress, 1, "h") ' https://www.hmailserver.com/forum/viewtopic.php?p=186868#p186868
		Exit Sub
	End If	

End Sub

Sub OnAcceptMessage(oClient, oMessage)

	'	Record entries for CatchSpam
	Dim spamDomain
	If (oMessage.HeaderValue("X-hMailServer-Spam") = "YES") Then 
		strRegEx = "[a-z0-9]+\.[a-z]{2,12}$"
		Set Matches = oLookup(strRegEx, oClient.HELO, False)
		For Each Match In Matches
		   spamDomain = Match.Value
		Next
		Call CatchSpam(spamDomain)
	End If

End Sub

palinka
Senior user
Senior user
Posts: 1393
Joined: 2017-09-12 17:57

Re: Ban spammers based on prior spamming

Post by palinka » 2019-12-05 16:16

I want to trap the spamassassin score because I think its a better predictor of true spam. Is spamassassin always "Reason-1" in the header?

Code: Select all

X-hMailServer-Reason-1: Tagged as Spam by SpamAssassin - (Score: 7)
Is there a better way to obtain the score?

palinka
Senior user
Senior user
Posts: 1393
Joined: 2017-09-12 17:57

Re: Ban spammers based on prior spamming

Post by palinka » 2019-12-05 20:31

Here's what I have in OnAcceptMessage. It seems to be working - at least I don't have any errors. Haven't received any spam yet since setting it up.

Code: Select all

	'	Record entries for CatchSpam
	Dim spamDomain, SAScore, SAHeader
	If oMessage.HeaderValue("X-hMailServer-Reason-1") <> "" Then 
		SAHeader = oMessage.HeaderValue("X-hMailServer-Reason-1")
		strRegEx = "[0-9]{1,3}"
		Set Matches = oLookup(strRegEx, SAHeader, False)
		For Each Match In Matches
		   SAScore = Match.Value
		Next
		If (CInt(SAScore) > 5) Then 
			strRegEx = "[a-z0-9]+\.[a-z]{2,12}$"
			Set Matches = oLookup(strRegEx, oClient.HELO, False)
			For Each Match In Matches
			   spamDomain = Match.Value
			Next
		End If
		Call CatchSpam(spamDomain)
	End If
The idea behind only using spamassassin score is that I get plenty of false positives from spamcop surbl. Spamassassin makes more sense when it comes to relying on a rejection policy.

palinka
Senior user
Senior user
Posts: 1393
Joined: 2017-09-12 17:57

Re: Ban spammers based on prior spamming

Post by palinka » 2019-12-06 21:23

This has been hard to test because I don't get much spam thanks to all my rejection filters, so I disabled them all. Found a little logic issue, fixed below:

OnAcceptMessage:

Code: Select all

	'	Record entries for CatchSpam
	Dim spamDomain, SAScore, SAHeader
	If oMessage.HeaderValue("X-hMailServer-Reason-Score") <> "" Then 
		SAHeader = oMessage.HeaderValue("X-hMailServer-Reason-Score")
		strRegEx = "[0-9]{1,3}"
		Set Matches = oLookup(strRegEx, SAHeader, False)
		For Each Match In Matches
		   SAScore = Match.Value
		Next
		If (CInt(SAScore) > 5) Then 
			strRegEx = "[a-z0-9]+\.[a-z]{2,12}$"
			Set Matches = oLookup(strRegEx, oClient.HELO, False)
			For Each Match In Matches
			   spamDomain = Match.Value
			Next
			EventLog.Write( "Spam Received: SA Score = " & CInt(SAScore) & ", HELO = " & oClient.HELO & " Domain = " & spamDomain )
			If spamDomain <> "" Then Call CatchSpam(spamDomain)
		End If
	End If
Moved "Call CatchSpam(spamDomain)" to within the IF statement for if SAScore > 5. That way it only gets called if SAScore > 5. Duh.... :roll:

Now I just have to wait for bona fide spammers to show up. Could be a few weeks.

Also, I was wrong about spamassassin score being only X-hMailServer-Reason-1 so I changed it to total spam score. But even if I get a FP hit from spamcop, the score (2) is too low to trigger by itself. In most cases, a spamassassin score will be required to push it over the (6) threshold. Most outliers would have already been rejected by my reject filters, so its unlikely anything other than a spamassassin score would get the total score up to 6 or higher. But.... even if you don't have a bunch of reject filters, a combined score of other tests should suffice to prevent hitting false positives.

palinka
Senior user
Senior user
Posts: 1393
Joined: 2017-09-12 17:57

Re: Ban spammers based on prior spamming

Post by palinka » 2019-12-11 15:58

Screenshot_20191211-081529_Brave.jpg
BAM! Gotcha, spammer!

First catch "in the wild". It worked exactly as intended.

This spammer was first seen yesterday and sent exactly 3 spams. I was a little disappointed because rejections don't begin until the 4th instance. Well, that happened this morning. As you can see, since my firewall ban tracks dropped connections, this spammer failed to send at least 12 spams. So far... :mrgreen:

Eventually this spammer will be listed at spamhaus. The whole point of this project is to trap the ones that haven't yet been listed at spamhaus, and it works very well.

palinka
Senior user
Senior user
Posts: 1393
Joined: 2017-09-12 17:57

Re: Ban spammers based on prior spamming

Post by palinka » 2019-12-12 15:45

Screenshot_20191212-084137_Brave.jpg
Continuing to work very well. 2 more IPs caught on the same model, as predicted, and 117 spams NOT received. :mrgreen:

Post Reply