Dump Junkmail For Unused Addresses

Use this forum if you have problems with a hMailServer script, such as hMailServer WebAdmin or code in an event handler.
Post Reply
User avatar
Snorkasaurus
Normal user
Normal user
Posts: 188
Joined: 2010-08-29 16:32
Location: Canada
Contact:

Dump Junkmail For Unused Addresses

Post by Snorkasaurus » 2014-07-24 04:30

Greetings Earthlings,

Looking through my logs [yes, I have no life] I see numerous attempts to deliver mail that are stopped by greylisting that I would like to stop before the greylist happens, and I would like to auto-create a temporary ban on the offending IP address. Over and over I see repeated attempts to deliver from fraud@aexp.com to email addresses I have never given to Amex. I also see tons of delivery attempts to addresses that have never existed at my domain (and by this I mean many attempts to deliver to a single non-existent address). It is nice that greylisting works, but I am a goof and don't even want to give them the courtesy of a 451 message.

This may seem like a waste of time but frankly greylisting is stopping almost all of my spam, and these few particular addresses are a significant percentage of it (and a significant portion of my port 25 traffic). My scripting is poor at best, but I have tried basic things like

Code: Select all

If (InStr(1, oMessage.Recipients(i).Address, "does.not.exist@mydomain.com", 1) > 0) Then
Result.Message = "Spam not accepted here."
Result.Value = 2
End If
yet I can't find a way to squeeze it in before the greylist happens. OnClientConnect is too early and OnSMTPData is too late. Even if I could get this to run before the message is accepted I really don't know how to change "Spam not accepted here." to "drop session and autoban". Is there any likelihood someone might be able to help me cut my log files in half?

Snork.

User avatar
mattg
Moderator
Moderator
Posts: 20123
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Dump Junkmail For Unused Addresses

Post by mattg » 2014-07-24 04:48

do you have a catch-all account set?
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
Snorkasaurus
Normal user
Normal user
Posts: 188
Joined: 2010-08-29 16:32
Location: Canada
Contact:

Re: Dump Junkmail For Unused Addresses

Post by Snorkasaurus » 2014-07-24 05:59

Hey mattg,

I do, however the messages I am referring to never make it to that mailbox. Actually, I just checked that mailbox and all six of the emails that landed in that account this month were destined for addresses that were at one time legitimate and intentionally deleted (implying that none were destined for the unwanted address combinations).

S.

percepts
Senior user
Senior user
Posts: 5282
Joined: 2009-10-20 16:33
Location: Sceptred Isle

Re: Dump Junkmail For Unused Addresses

Post by percepts » 2014-07-24 12:14

post a section of log where several of these "spam" mails are arriving

percepts
Senior user
Senior user
Posts: 5282
Joined: 2009-10-20 16:33
Location: Sceptred Isle

Re: Dump Junkmail For Unused Addresses

Post by percepts » 2014-07-24 13:22

also

in the received mail headers what is in Return-Path ? Is it always fraud@aexp.com or is it something else and if so what ?

User avatar
SorenR
Senior user
Senior user
Posts: 3184
Joined: 2006-08-21 15:38
Location: Denmark

Re: Dump Junkmail For Unused Addresses

Post by SorenR » 2014-07-24 14:34

Maybe an idea...

In the database, in table hm_greylisting_triplets there are two columns; glipaddress1 and glrecipientaddress...

Something like...

Code: Select all

select INET_NTOA(glipaddress1) AS ip_str from hm_greylisting_triplets 
where glrecipientaddress = "bogus@my.domain";
and then build an auto-ban thingy with the IP address returned by "ip_str" - if possible... :?:

or...

Code: Select all

select glrecipientaddress from hm_greylisting_triplets 
where glipaddress1 = INET_ATON("111.222.333.444");
Use in OnClientConnect() to deny connect if "glrecipientaddress" = "bogus@my.domain" ...
NOTE.. Unused triplets (glpassedcount = 0) will expire over time, so it's not foolproof :oops:

(MySQL) INET_ATON() = IP Address -> Numeric conversion
(MySQL) INET_NTOA() = Numeric -> IP Address conversion
SørenR.

“With age comes wisdom, but sometimes age comes alone.”
- Oscar Wilde

User avatar
Snorkasaurus
Normal user
Normal user
Posts: 188
Joined: 2010-08-29 16:32
Location: Canada
Contact:

Re: Dump Junkmail For Unused Addresses

Post by Snorkasaurus » 2014-07-24 16:09

Hey percepts,

Here's a common log file example:

Code: Select all

"SMTPD"	3936	2157	"2014-07-23 23:34:37.057"	"190.29.138.62"	"SENT: 220 mx1.snork.ca ESMTP"
"SMTPD"	3936	2157	"2014-07-23 23:34:37.229"	"190.29.138.62"	"RECEIVED: EHLO static-adsl190-29-141-23.une.net.co"
"SMTPD"	3936	2157	"2014-07-23 23:34:37.229"	"190.29.138.62"	"SENT: 250-mx1.snork.ca[nl]250-SIZE 20480000[nl]250 AUTH LOGIN"
"SMTPD"	3936	2157	"2014-07-23 23:34:37.369"	"190.29.138.62"	"RECEIVED: MAIL FROM:<ussnorkleka@une.net.co>"
"SMTPD"	3936	2157	"2014-07-23 23:34:37.385"	"190.29.138.62"	"SENT: 250 OK"
"SMTPD"	3936	2157	"2014-07-23 23:34:37.541"	"190.29.138.62"	"RECEIVED: RCPT TO:<ussnorklek@snork.ca>"
"SMTPD"	3936	2157	"2014-07-23 23:34:37.572"	"190.29.138.62"	"SENT: 451 Please try again later."
"SMTPD"	3936	2157	"2014-07-23 23:34:37.729"	"190.29.138.62"	"RECEIVED: QUIT"
"SMTPD"	3936	2157	"2014-07-23 23:34:37.729"	"190.29.138.62"	"SENT: 221 goodbye"
The combination of letters "ussnorklek" appear quite frequently in both FROM and TO addresses and are a word I have never used. And another common occurrence:

Code: Select all

"SMTPD"	3936	2049	"2014-07-23 11:54:14.151"	"118.69.81.16"	"SENT: 220 mx1.snork.ca ESMTP"
"SMTPD"	3936	2049	"2014-07-23 11:54:14.463"	"118.69.81.16"	"RECEIVED: EHLO [118.69.81.16]"
"SMTPD"	3936	2049	"2014-07-23 11:54:14.463"	"118.69.81.16"	"SENT: 250-mx1.snork.ca[nl]250-SIZE 20480000[nl]250 AUTH LOGIN"
"SMTPD"	3936	2049	"2014-07-23 11:54:14.807"	"118.69.81.16"	"RECEIVED: MAIL FROM: <fraud@aexp.com> BODY=7BIT"
"SMTPD"	3936	2049	"2014-07-23 11:54:14.807"	"118.69.81.16"	"SENT: 250 OK"
"SMTPD"	3936	2049	"2014-07-23 11:54:15.119"	"118.69.81.16"	"RECEIVED: RCPT TO:<address1@snork.ca>"
"SMTPD"	3936	2049	"2014-07-23 11:54:15.151"	"118.69.81.16"	"SENT: 250 OK"
"SMTPD"	3936	2049	"2014-07-23 11:54:15.463"	"118.69.81.16"	"RECEIVED: RCPT TO:<address2@snork.ca>"
"SMTPD"	3936	2049	"2014-07-23 11:54:15.463"	"118.69.81.16"	"SENT: 451 Please try again later."
Where address1 and address2 actually did exist as aliases on the server but have since been deleted (they were just disposable addresses). As you can see the greylisting is 451'ing them, and so a Return-Path is never presented.

@SorenR: for that to work I would still need the greylist to take place right? I am hoping for more along the lines of an event between OnClientConnect and OnSMTPData where I can act upon the MAIL FROM and RCPT TO commands. Or is that maybe the wrong way to look at it?

I will admit that what I am looking for doesn't provide much more security or "spam fighting power" than the greylist feature alone, but it would be nice to be able to act on these entries before the greylist so I can just stop accepting their traffic.

S.

percepts
Senior user
Senior user
Posts: 5282
Joined: 2009-10-20 16:33
Location: Sceptred Isle

Re: Dump Junkmail For Unused Addresses

Post by percepts » 2014-07-24 17:18

A couple of options

Code: Select all

Sub OnClientConnect(oClient)
  If (oClient.IPAddress = "118.69.81.16" ) Then 
   Result.Value = 1
  End if
End Sub
Obviously above requires you to know their IP number in advance. And note there is no option to send message back. This best option if possible (i.e. you have the IP)

Code: Select all

Sub OnSMTPData(oClient, oMessage)
 If (oMessage.FromAddress = "fraud@aexp.com") Then
   Result.Value = 1
 End if
End Sub
FromAddress is normally what you would see in Return-Path which is why I asked. But it may be spoofed I think. You can try. However, I'm not sure if greylisting checks take place before OnSMTPData. You will have to try. FromAddress is populated in OnSMTPData. From isn't.

Setting Result.Value to 1 will reject the mail without a message which effectively drops the session.

first option rejects the initail connect and works just like an autoban. Nothing in logs.

p.s. if you know the IP then you can add a block for it in your firewall or router. That way it never even reaches hmailserver.

User avatar
SorenR
Senior user
Senior user
Posts: 3184
Joined: 2006-08-21 15:38
Location: Denmark

Re: Dump Junkmail For Unused Addresses

Post by SorenR » 2014-07-24 17:52

Snorkasaurus wrote:@SorenR: for that to work I would still need the greylist to take place right? I am hoping for more along the lines of an event between OnClientConnect and OnSMTPData where I can act upon the MAIL FROM and RCPT TO commands. Or is that maybe the wrong way to look at it?

I will admit that what I am looking for doesn't provide much more security or "spam fighting power" than the greylist feature alone, but it would be nice to be able to act on these entries before the greylist so I can just stop accepting their traffic.

S.
Well, Greylisting is sort of first encounter with the "alien" without actually receiving the email, you know "from where & whom" and "to whom" and you get a second chance to do something if/when they try to connect again... :wink:

Sort of...

Connect -> New Greylist triplet
Connect -> OnClientConnect -> Known from triplet -> Reject
Connect -> OnClientConnect -> Known from triplet -> Reject
...
Connect -> OnClientConnect -> Known from triplet -> Reject
Connect -> OnClientConnect -> Known from triplet -> Reject
Connect -> Triplet expired -> New Greylist triplet
Connect -> OnClientConnect -> Known from triplet -> Reject
Connect -> OnClientConnect -> Known from triplet -> Reject
...
Connect -> OnClientConnect -> Known from triplet -> Reject
Connect -> OnClientConnect -> Known from triplet -> Reject
Connect -> Triplet expired -> New Greylist triplet
...
etc...

Note that if you receive both valid AND "invalid" emails from the same source, this will not work as it will also reject valid emails - BUT - since the rejection is already at OnClientConnect, one may hope to fool them into removing your address as "there is no server responding" ...

I know it's sort of an unorthodox solution and some of my solutions do not always work out as intended... but that's how my brain works :mrgreen:
SørenR.

“With age comes wisdom, but sometimes age comes alone.”
- Oscar Wilde

User avatar
Snorkasaurus
Normal user
Normal user
Posts: 188
Joined: 2010-08-29 16:32
Location: Canada
Contact:

Re: Dump Junkmail For Unused Addresses

Post by Snorkasaurus » 2014-07-24 18:07

@percepts:
Unfortunately I don't have the IP addresses ahead of time, and the activity seems to be coming from numerous different and changing IP's. I admit that I haven't actually checked, but my guess is that they are mostly virus-infected zombie PC's. :-(

I have tried something like that in OnSMTPData, but it takes place immediately after the DATA command is sent by the remote system, which is never reached in these cases because of the greylisting. I have a feeling that without an event check just after the RCPT TO command (and just before greylisting happens) there won't be a way to trap these.

@SorenR:
SorenR wrote:Note that if you receive both valid AND "invalid" emails from the same source, this will not work as it will also reject valid emails
I really don't think that any of these IP's are also sending legitimate email, but I would prefer to behave conservatively and ... hmmm, now that I think of it, if I were to dump/tempban an IP for delivering to ussnorklek@snork.ca then someone could block gmail from me by simply sending an email to that address from a legitimate gmail account. Methinks that a mountain of greylisting logs might still be in my future.

S.

User avatar
SorenR
Senior user
Senior user
Posts: 3184
Joined: 2006-08-21 15:38
Location: Denmark

Re: Dump Junkmail For Unused Addresses

Post by SorenR » 2014-07-24 18:25

Snorkasaurus wrote:Methinks that a mountain of greylisting logs might still be in my future.

S.
Logging is good ... :mrgreen:

I'm working on a project with my server to rid myself of some of the "Internet Noise" out there... Somewhere between 300-1100 lines every day on a server with 7 users... :roll:

Rejecting connections based on Continent, Country and now also ISP.

http://www.hmailserver.com/forum/viewto ... 75#p163275
SørenR.

“With age comes wisdom, but sometimes age comes alone.”
- Oscar Wilde

User avatar
Snorkasaurus
Normal user
Normal user
Posts: 188
Joined: 2010-08-29 16:32
Location: Canada
Contact:

Re: Dump Junkmail For Unused Addresses

Post by Snorkasaurus » 2014-07-24 18:46

Don't get me wrong, I like logging... I just want to remove the 80% or more of my logs that are for greylisting. ;-)

I think my solution might be to maintain a primary hMailServer that handles real mail and client activity while using a secondary hMailServer that acts as a backup MX and handles all the greylisting for me. It seems a bit of a waste to setup an extra hMailServer box just for greylisting but it sure keeps the traffic off the "real" server!

I am doing something similar to your geo-blocking with iptables and ipset on a debian-based firewall. The thing I like about doing it at the firewall is that I can control access to other things as well (such as ports 80 and 443).

S.

User avatar
SorenR
Senior user
Senior user
Posts: 3184
Joined: 2006-08-21 15:38
Location: Denmark

Re: Dump Junkmail For Unused Addresses

Post by SorenR » 2014-07-24 19:03

Snorkasaurus wrote:Don't get me wrong, I like logging... I just want to remove the 80% or more of my logs that are for greylisting. ;-)

I think my solution might be to maintain a primary hMailServer that handles real mail and client activity while using a secondary hMailServer that acts as a backup MX and handles all the greylisting for me. It seems a bit of a waste to setup an extra hMailServer box just for greylisting but it sure keeps the traffic off the "real" server!

I am doing something similar to your geo-blocking with iptables and ipset on a debian-based firewall. The thing I like about doing it at the firewall is that I can control access to other things as well (such as ports 80 and 443).

S.
Ah... Oh, that reminds me... I need to go get a new PSU for my Cisco ASA 5500... :oops:

On the subject of Greylisting and Backup MX'es... I do Greylisting and have a Backup MX at my ISP... First the "aliens" connect to my mailserver, become Greylisted, then they connect to my Backup MX and the email is delivered via the Backup MX. Both regular emails and SPAM emails... :roll:
So much for Greylisting :evil:

I've changed my external DNS today so MX records are now; priority 10 (preferred) - hMailserver, priority 20 - Backup MX and priority 30 - hMailServer.

Since last restart of server (24+ hours ago); emails: 816, virus: 21 and SPAM: 258 (plus whatever is on my blacklist rules).
The 816 emails include the copies I forward to my spam account, and also the blacklisted emails, so in reality less than half ~350...
SørenR.

“With age comes wisdom, but sometimes age comes alone.”
- Oscar Wilde

User avatar
mattg
Moderator
Moderator
Posts: 20123
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Dump Junkmail For Unused Addresses

Post by mattg » 2014-07-25 00:45

mattg wrote:do you have a catch-all account set?
Snorkasaurus wrote:Hey mattg,

I do, however the messages I am referring to never make it to that mailbox. Actually, I just checked that mailbox and all six of the emails that landed in that account this month were destined for addresses that were at one time legitimate and intentionally deleted (implying that none were destined for the unwanted address combinations).
I guess my point was, why don't you remove the catch-all

That'll reject this unwanted mail first up.

You could still set an alias for each 'one time legitimate and intentionally deleted' addresses or use plus addressing

To me, this sounds like you want the catch-all to be selective, which it can't be, and so you find an alternative to the catch-all
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
Snorkasaurus
Normal user
Normal user
Posts: 188
Joined: 2010-08-29 16:32
Location: Canada
Contact:

Re: Dump Junkmail For Unused Addresses

Post by Snorkasaurus » 2014-07-25 03:33

Hey mattg,

Hmmm, perhaps I should have elaborated... I do have a catch-all on the domain, but I do not have a catch-all on this backup MX server that is now essentially acting as a "greylisting/backup MX" machine. Even if I do disable the catch-all address I still get:

Code: Select all

"SMTPD"	1884	8144	"2014-07-24 21:24:10.087"	"69.196.173.68"	"SENT: 220 mail.snork.ca ESMTP NO UCE WASSUP"
"SMTPD"	1884	8144	"2014-07-24 21:24:22.009"	"69.196.173.68"	"RECEIVED: helo there"
"SMTPD"	1884	8144	"2014-07-24 21:24:22.009"	"69.196.173.68"	"SENT: 250 Hello."
"SMTPD"	1920	8144	"2014-07-24 21:24:35.900"	"69.196.173.68"	"RECEIVED: mail from: guy@place.org"
"SMTPD"	1920	8144	"2014-07-24 21:24:35.900"	"69.196.173.68"	"SENT: 250 OK"
"SMTPD"	1920	8144	"2014-07-24 21:24:45.587"	"69.196.173.68"	"RECEIVED: rcpt to: ussnorklek@snork.ca"
"SMTPD"	1920	8144	"2014-07-24 21:24:45.587"	"69.196.173.68"	"SENT: 550 Unknown user"
"SMTPD"	1920	8144	"2014-07-24 21:25:03.415"	"69.196.173.68"	"RECEIVED: quit"
"SMTPD"	1920	8144	"2014-07-24 21:25:03.415"	"69.196.173.68"	"SENT: 221 goodbye"
which unfortunately doesn't cut down on the lines of logging, and doesn't let me tempban them. In fact, greylisting is stopping so much of my spam that I have dropped the MX record for my primary mail server and just use a simple backup mail server as my sole inbound system. My primary server is down to less than 50k of logs per day and my backup MX logs are easily over 80% greylist entries (possibly over 90%).

S.

percepts
Senior user
Senior user
Posts: 5282
Joined: 2009-10-20 16:33
Location: Sceptred Isle

Re: Dump Junkmail For Unused Addresses

Post by percepts » 2014-07-25 18:06

If you haven't already considered this script then its definitely worth considering. It's highly effective but it's not for everyone. Just depends whether you expect foreign email and where from. If not then use it.

http://www.hmailserver.com/forum/viewto ... 20&t=16679

It stops connection in OnClientConnect and only leaves one line in event log for anything rejected. So will reject before any antispam or virus checks.

Just make sure you have CA (Canada), US and DE open at a minimum. DE is for mail.hmailserver.com traffic from this forum.

User avatar
Snorkasaurus
Normal user
Normal user
Posts: 188
Joined: 2010-08-29 16:32
Location: Canada
Contact:

Re: Dump Junkmail For Unused Addresses

Post by Snorkasaurus » 2014-07-25 19:41

Hey percepts,

Thanks for the link, it looks pretty cool. I am actually already blocking some countries by using lists from IPDeny and some ipset rules on my firewall (which gives me the added bonus of being able to block them from other services like http at the same time). If anyone cares to see how, I'd be happy to share it (as long as nobody minds the fact that it isn't hMailServer specific).

S.

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: Dump Junkmail For Unused Addresses

Post by ^DooM^ » 2014-07-25 19:44

You could put a script together that runs on a scheduled task to pull the rejected ip's from greylisting table and insert these into auto ban table. Would need to have some sanity checks built in to take into account local ip ranges and routes if setup but could be done.
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

percepts
Senior user
Senior user
Posts: 5282
Joined: 2009-10-20 16:33
Location: Sceptred Isle

Re: Dump Junkmail For Unused Addresses

Post by percepts » 2014-07-25 20:35

Snorkasaurus wrote:Hey percepts,

Thanks for the link, it looks pretty cool. I am actually already blocking some countries by using lists from IPDeny and some ipset rules on my firewall (which gives me the added bonus of being able to block them from other services like http at the same time). If anyone cares to see how, I'd be happy to share it (as long as nobody minds the fact that it isn't hMailServer specific).

S.
you can use maxminds file to block from http too. Depending on http server you use but with apache 2.2 on windows I implemented Mod Geoip from

http://www.apachehaus.com/cgi-bin/download.plx

this provides a couple of PHP enviroment variables (country) which are globally available in PHP.

But I also download maxminds csv file, run an sql select on country and use powershell to create firewall rules allowing only those IPs I want into my websites. Also maintain some other firewall rules to allow a few search engine bots through.

User avatar
Snorkasaurus
Normal user
Normal user
Posts: 188
Joined: 2010-08-29 16:32
Location: Canada
Contact:

Re: Dump Junkmail For Unused Addresses

Post by Snorkasaurus » 2014-07-25 21:23

Hey percepts,
percepts wrote:But I also download maxminds csv file, run an sql select on country and use powershell to create firewall rules allowing only those IPs I want into my websites. Also maintain some other firewall rules to allow a few search engine bots through.
I much prefer blocking inbound "bad traffic" at the firewall when I can rather than adding anything to an internal application... can I ask what kind of firewall you have? Is it iptables/ipset based?

S.

percepts
Senior user
Senior user
Posts: 5282
Joined: 2009-10-20 16:33
Location: Sceptred Isle

Re: Dump Junkmail For Unused Addresses

Post by percepts » 2014-07-25 21:50

windows firewall that comes with Windows 7

User avatar
Snorkasaurus
Normal user
Normal user
Posts: 188
Joined: 2010-08-29 16:32
Location: Canada
Contact:

Re: Dump Junkmail For Unused Addresses

Post by Snorkasaurus » 2014-07-25 23:27

Hey guys,

I think I may have found something that I can live with. The last few messages got me thinking... why not try to block this at the firewall level? After a little testing I came up with this:

Code: Select all

iptables -I FORWARD -p tcp --dport 25 -m string --string 'rcpt to: crappy.address@mydomain.ca' --icase --algo bm -j REJECT
Advantages: If someone sends a "bad" message via a legitimate mail server (trying to block me from receiving legitimate mail) the session will be dumped but subsequent real messages will still be able to get through. hMailServer doesn't have to care about good or bad addresses, or even non-existent ones - the session just disappears and times out.
Disadvantages: I don't get to see the "offending command" in my hMailServer logs. The router doesn't actually pass the packet through so hMailServer can't reply and/or log the entry... I guess this means I should be pretty careful about the strings I put in there.

Code: Select all

"SMTPD"	1884	8669	"2014-07-25 17:09:32.430"	"69.196.173.68"	"SENT: 220 mail.snork.ca ESMTP NO UCE WASSUP"
"SMTPD"	1884	8669	"2014-07-25 17:09:37.133"	"69.196.173.68"	"RECEIVED: helo there"
"SMTPD"	1884	8669	"2014-07-25 17:09:37.133"	"69.196.173.68"	"SENT: 250 Hello."
"SMTPD"	1920	8669	"2014-07-25 17:09:49.696"	"69.196.173.68"	"RECEIVED: mail from: guy@place.org"
"SMTPD"	1920	8669	"2014-07-25 17:09:49.696"	"69.196.173.68"	"SENT: 250 OK"
I tried setting up tarpitting as an iptables destination, but it seems Debian/iptables needs a whack of software to support that and even then I couldn't get it to work in my testing. If anyone knows how to make iptables accept the current packet and then reset the session I would love to hear it though I will admit we have strayed from hMailServer to some significantly different territory.

S.

percepts
Senior user
Senior user
Posts: 5282
Joined: 2009-10-20 16:33
Location: Sceptred Isle

Re: Dump Junkmail For Unused Addresses

Post by percepts » 2014-07-25 23:32

glad you got it sorted. Just so long as it doesn't leave hmail with open connections or processes running for the session I guess it should be fine.

Post Reply