Support for ETRN and ATRN [90%]

layer · Post by **layer** » 2004-09-19 03:36

Can be implemented ?
i think is easy

thanks

MOD EDIT: ETRN 90% done (fully functional but needs proper GUI area)

Post by **martin** » 2004-09-19 17:43

What do you want to use it for?

layer · Post by **layer** » 2004-09-19 17:46

martin wrote:What do you want to use it for?

for secondary MX record or Backup-MX...."to release queue early"

Post by **martin** » 2004-09-19 17:50

Not sure what you mean now.

Version 3.3 has something called Routes. These can be used if you want to have a server running as a mail backup server. Whats missing?

laf · Post by **laf** » 2004-09-19 22:53

What he wants is to use Hmail as a backup MX, if this backup MX accepts mail he wants the primary MX record (usually an exchange server) to be able to issue an ETRN command to the Hmail server to say hey i'm ready to accept mail again, flush anything you have in your queue to me now.

Neil

Post by **martin** » 2004-09-19 23:18

It's not very easy to implement support for this. I'll think some about it and will probably add it in 3.4.

chrix · Post by **chrix** » 2006-01-10 11:10

Hi Martin !

Is ETRN/ATRN always on your roadmap ? (http://www.faqs.org/rfcs/rfc1985.html)

chrix · Post by **chrix** » 2006-01-11 18:23

afflicted to insist, but this function is of an extraordinary utility.

Here how I currently make use of it: I have a personal mailserver with a domain name and a friend has a waiter with another domain name. With crossed MX records, if one of our mailserver is broken down, emails entering are stored on the other one, and it come again online, thanks to the ETRN, it recovers them. It, isn't this is useful?

So, the same question : will the ETRN be integrated into hMailServer ?

Post by **martin** » 2006-01-12 01:15

This feature is handled like the others. If enough people votes for it, it will defintivly be implemented. I like the idéa myself..

chrix · Post by **chrix** » 2006-01-12 12:07

ETRN must be vote before being implemented ? So I'll offer a weekend of holidays at Monaco for each voter !!!

But I you like this idea, perhaps it could rise up earlier instead of waiting much voters... Nope ?

CraigHarris · Post by **CraigHarris** » 2006-01-12 23:34

martin wrote:It's not very easy to implement support for this.

Can't you just add a flag for "Awaiting ETRN", in which case the message gets totally ignored on all delivery queue operations -- this gets set if the unavailable server is listed as supporting such -- servers get added to that list, either manually or if support is determined from any SMTP conversation with such a server.

Then if the SMTP conversation recieves an ETRN it clears the flag, and sets next retry to the current timestamp.

Ideally there should be an option to "force check every X hours" - just in-case the reply doesn't get sent.

hmmm... ok, there are a few things to implement and other features are more important -- actually I think it's about time I downloaded the source code and played arround with it anyway.

Sippin45 · Post by **Sippin45** » 2006-03-14 02:44

If I use hMS Routes will mails pass thru or bypass spam protection?

CraigHarris · Post by **CraigHarris** » 2006-03-14 17:03

Routes don't affect spam protection, though the behaviour of routes varies whether they are set to behave as local or external domains (for purposes of authentication)

westdam · Post by **westdam** » 2006-10-27 18:11

mmm
with the route function these commands are insteresting..

iprat · Post by **iprat** » 2006-11-09 11:41

IMO this function might be very interesting for Martin's plans of giving some redundancy tools to hMailServer, as this is an official way of comunicating mail servers, isn't it ?

This should help to exchange data between servers when they go online again after a connection, power, or even OS failure.

westdam · Post by **westdam** » 2006-11-09 13:25

yep i agree with iprat..

motiv8d · Post by **motiv8d** » 2007-02-14 03:33

Yes this feature would definitely be helpful. At the moment I have the backup hmailserver route smtp delivery set to 5 mins x 999 times to try to get around not having a dequeue feature.
The way I manually dequeue mail from my backup ETRN mail server is with a great auto telnet utility called tst10. It reads a script and dequeues all mail for the respective domain/subdomain in one hit instead of it coming in dribs and drabs.

The script goes like:
mx2.backupdomain.com 25
WAIT ")"
SEND "ETRN mydomain.com\m"
WAIT "started"
SEND "ETRN subdomain.mydomain.com\m"
WAIT "started"
quit

Simple and very effective.

Sheff · Post by **Sheff** » 2008-08-21 23:15

Sorry to be an arse and bump a 12 month old topic ... but did this ever happen and if so ... how do you go about setting it up?

humble apologies if im being a bit dense ... just couldnt see anywhere in 4.4.1 that mentioned ETRN and Martin mentioned 3.4 below ... which probably means I'm being dense ...

Cheers

Sheff.

Post by **martin** » 2008-08-21 23:38

Things under feature requests are not implemented. They are just still requests.
I implement the features which highest voting score...

Sheff · Post by **Sheff** » 2008-08-21 23:45

ah k ... sorry ... didnt know the requests got deleted once they'd been implimented

Any idea how far up (or not) the request list this is?

Thanks

Ian

westdam · Post by **westdam** » 2008-08-31 12:36

mmm anyway the ETRN and ATRN feature are still " tricky" to implement and works nice only with the same SMTP Server on each side even if they're RFC compliant..

lots of trouble comes out unfortunately with ATRN/ETRN . Unfortunately 'couse I like the idea but it's doesnt work as I want..

Wimbo · Post by **Wimbo** » 2009-07-29 00:51

martin wrote:It's not very easy to implement support for this. I'll think some about it and will probably add it in 3.4.

It would appear that this hasn't happened yet. Off hand, I can't think of an MTA that I've used that doesn't offer this feature. It's pretty simple in concept:

1 Provide a place to list domains for which relay service is provided. In at least one case, the HOSTS file is used for this. This list contains the domain name and IP address for delivery. Where the IP address is dynamic, a value of 0.0.0.0 would indicate "unspecified".

2 Add logic to message acceptance process to accept messages for a domain in the relay list, even if there are no local mailboxes on hMS. These messages simply stay in the queue until fetched, delivered, or expire in the normal way.

3 When an ETRN request is received, fire off an SMTP queue process to select all messages in the queue for the requested domain and send them immediately.

It's most commonly used for domains that are intermittently connected, for example a MTA on the end of a dial-up connection. Also useful for back-up purposes.

William

Post by **martin** » 2009-07-29 17:30

"Probably" is indeed a vague term.

1 Provide a place to list domains for which relay service is provided. In at least one case, the HOSTS file is used for this. This list contains the domain name and IP address for delivery. Where the IP address is dynamic, a value of 0.0.0.0 would indicate "unspecified".

This already exists in hMailServer. Except for the "dynamic IP address" part.

2 Add logic to message acceptance process to accept messages for a domain in the relay list, even if there are no local mailboxes on hMS. These messages simply stay in the queue until fetched, delivered, or expire in the normal way.

Also exists.

3 When an ETRN request is received, fire off an SMTP queue process to select all messages in the queue for the requested domain and send them immediately.

Does not exist.

I'm not saying it's impossible or anything, just that it's more than a hours worth of work. For instance, a single message may be addressed to 10 different recipients. When an ETRN is issued, hMailServer should do a delivery attempts on those messages, but only for recipients which matches the domain.

Wimbo · Post by **Wimbo** » 2009-07-29 20:57

martin wrote:"Probably" is indeed a vague term.

This already exists in hMailServer. Except for the "dynamic IP address" part
...
Also exists.

Cool, and cool

3 When an ETRN request is received, fire off an SMTP queue process to select all messages in the queue for the requested domain and send them immediately.

Does not exist.

I'm not saying it's impossible or anything, just that it's more than a hours worth of work. For instance, a single message may be addressed to 10 different recipients. When an ETRN is issued, hMailServer should do a delivery attempts on those messages, but only for recipients which matches the domain.

I don't think that there's any need to be that fussy. I'm not aware of any other MTA that bothers, either. Make a delivery attempt for all messages to be delivered to that domain. If some messages are also to be delivered elsewhere, then their delivery attempt count simply gets incremented by 1. One out of up to 360 (One attempt every 20 minutes for 5 days for example, as required by RFC 1822 IIRC) isn't going to make any difference in the real world.

HTH.

William

Post by **martin** » 2009-07-29 21:28

> isn't going to make any difference in the real world.

That may be right.

> One out of up to 360 (One attempt every 20 minutes for 5 days for example, as required by RFC 1822 IIRC)

Hm. RFC 1822 seems not to be related to this?

Wimbo · Post by **Wimbo** » 2009-07-29 21:53

>> One out of up to 360 (One attempt every 20 minutes for 5 days for example, as required by RFC 1822 IIRC)

> Hm. RFC 1822 seems not to be related to this?

Sorry, you're quite right. It's RFC 1821, section 4.5.4.1, which says in part: -

The sender MUST delay retrying a particular destination after one
attempt has failed. In general, the retry interval SHOULD be at
least 30 minutes; however, more sophisticated and variable strategies
will be beneficial when the SMTP client can determine the reason for
non-delivery.

Retries continue until the message is transmitted or the sender gives
up; the give-up time generally needs to be at least 4-5 days. The
parameters to the retry algorithm MUST be configurable.

30 minutes x 5 days = 240 attempts - in any case, a far larger number than hMailServer's default of 4 attempts! I work on 144 attempts by default.

William

Post by **martin** » 2009-07-29 22:01

>>> One out of up to 360 (One attempt every 20 minutes for 5 days for example, as required by RFC 1822 IIRC)

>> Hm. RFC 1822 seems not to be related to this?

> Sorry, you're quite right. It's RFC 1821, section 4.5.4.1, which says in part: -

RFC 1821 is "Integration of Real-time Services in an IP-ATM Network".

RFC 2821 is the right way.

(A minor addition. 4-5 days isn't really a requirements. The only requirement in that snippet is that the retry algorithm must be configurable. SHOULD in RFC's is a synonym for "RECOMMENDED" according to RFC 2119, and recommendations can't really be "requirements".)

One big benefit with having a low retry count is that users are more quickly notified when a message delivery fails. It may be annoying to people and lead to confusion if it takes 48 hours before they get a bounce message back. Of course, hMailServer could be nice to users and tell them that hMailServer is still trying to deliver the message, but this isn't implemented, yet.

Bill48105 · Post by **Bill48105** » 2010-11-02 02:33

Drum roll please... 6 years in the making based on the original post above but I have added ETRN to 5.4 alpha!! WOO WOO!

Server-side when a message fails to be delivered the destination is checks to see if it is defined as Route domain. If it is then the message is changed to HOLD status in the queue. It is held there INDEFINITELY unless either 1. Admin does "Send now" or "Delete" in admin or ETRN thedomainname.com is received via SMTP.

Client-side is a touch of a kludge for now as I was not up to all the needed changes so I modified "External accounts" section to be used for ETRN as well. I know, I know not exactly pretty but it works.

To setup ETRN client (that sends the ETRN command to a remote server on a given interval) you add a new external account being sure the "Name" field is exactly "ETRN yourdomain.com", set remote SMTP as host, likely port 25 for port, obviously you want it enabled, set # of minutes you desire & ignore the rest of the settings as they are not used for ETRN.

Sample server-side logs

"SMTPD" 3820 9 "2010-11-01 19:24:11.047" "127.0.0.1" "SENT: 220 AE7FB352FC4D4B7 ESMTP"
"SMTPD" 3180 9 "2010-11-01 19:24:11.047" "127.0.0.1" "RECEIVED: HELO mydomain.com"
"SMTPD" 3180 9 "2010-11-01 19:24:11.057" "127.0.0.1" "SENT: 250 Hello."
"SMTPD" 2116 9 "2010-11-01 19:24:11.057" "127.0.0.1" "RECEIVED: ETRN mydomain.com"
"SMTPD" 2116 9 "2010-11-01 19:24:11.057" "127.0.0.1" "SMTPDeliverer - ETRN - Route found, continuing.."
"SMTPD" 2116 9 "2010-11-01 19:24:11.057" "127.0.0.1" "SMTPDeliverer - ETRN - Route settings read successfully."
"SMTPD" 2116 9 "2010-11-01 19:24:11.057" "127.0.0.1" "SENT: 250 OK, message queuing started for mydomain.com"
"SMTPD" 2116 9 "2010-11-01 19:24:11.057" "127.0.0.1" "SMTPDeliverer - ETRN - 250 OK, message queuing started."
"SMTPD" 1232 9 "2010-11-01 19:24:11.087" "127.0.0.1" "RECEIVED: QUIT"
"SMTPD" 1232 9 "2010-11-01 19:24:11.087" "127.0.0.1" "SENT: 221 goodbye"

Sample client-side logs

"SMTPC" 3180 8 "2010-11-01 19:24:11.047" "127.0.0.1" "RECEIVED: 220 AE7FB352FC4D4B7 ESMTP"
"SMTPC" 3180 8 "2010-11-01 19:24:11.047" "127.0.0.1" "SENT: HELO mydomain.com"
"SMTPC" 1232 8 "2010-11-01 19:24:11.057" "127.0.0.1" "RECEIVED: 250 Hello."
"SMTPC" 1232 8 "2010-11-01 19:24:11.057" "127.0.0.1" "SENT: ETRN mydomain.com"
"SMTPC" 2116 8 "2010-11-01 19:24:11.087" "127.0.0.1" "RECEIVED: 250 OK, message queuing started for mydomain.com"
"SMTPC" 2116 8 "2010-11-01 19:24:11.087" "127.0.0.1" "SENT: QUIT"
"SMTPC" 3416 8 "2010-11-01 19:24:11.107" "127.0.0.1" "RECEIVED: 221 goodbye"

Do note that hMail stops you from setting things up so that hmail will connect to itself but my local version has that restriction disabled for testing. I can't imagine there'd be a legit reason for you to ever ETRN yourself anyway but for the sake of testing & posting up sample logs I did.

So.. Who besides me is excited about this addition?

Bill
NOTE: The build available for download as of today that martin posted recently DOES NOT HAVE ETRN!
I am just posting up the good news in advance of it being included officially to give those looking for it something to look forward to.

siliconbullet · Post by **siliconbullet** » 2011-05-22 11:56

I am interested in using ETRN for various client SMTP backup scenarios.

I cannot seem to see a download yet for the 5.4 beta, am I missing something? Do we have an ETA for 5.4 becoming a beta download or the latest stable version?

Thanks.

Bill48105 · Post by **Bill48105** » 2011-05-24 21:48

siliconbullet,
5.4 has had ETRN since after the initial alpha (in other words all except the 1st alpha release). You can download the latest at:
http://www.hmailserver.com/?page=download_latest
Since the GUI hasn't caught up to the server yet the setup of ETRN leaves a bit to be desired but my priority was to get working ETRN in figuring those who need it are willing to put a little effort into configuration at this point until the GUI can be updated. I'll post up an updated how-to here soon.
Bill

Bill48105 · Post by **Bill48105** » 2011-05-24 23:03

Basic ETRN functionality is in hmailserver 5.4 following the initial alpha release. Functionality was focused on rather than the GUI so as such at this point ETRN is considered an advanced 'unsupported' feature but I wanted to document how to use it in this how-to for those interested & willing (capable) of the effort needed to use.

It is important to note you DO NOT do both the client & server sides on the same server for the same domain!! If you are interested in ETRN it is assumed you understand its purpose and if you don't understand what I mean by that odds are ETRN is not for you..

The server side of ETRN acts as an extension to mail Routes used for "mx backups" also called secondary domains etc. When a Route is created for a domain & there is ever a problem sending email to that domain hmailserver will normally follow the normal retry schedule, tying up the queue. With ETRN, retries of that domains are DISABLED until an ETRN request for that domain is made. SUPER IMPORTANT TO READ THAT AGAIN: In other words, if you enabled ETRN for a Route domain if a failure to deliver ever occurs ALL FUTURE EMAIL WILL BE HELD INDEFINITELY unless a remote server connects & issues ETRN thedomain.com to tell hmailserver to release that email. Once that command is issued held email will be added back into the queue & delivery will be attempted like normal. I can't stress strongly enough how important it is to understand that fact as if ETRN commands don't happen email for that domain could be held FOREVER.

The client side of ETRN simply connects to a remote server, issues a HELO yourdomain.com then issues ETRN yourdomain.com and disconnects on a given interval. In a way this acts as a 'heartbeat' to show the other server it is 'alive and well' but in reality it is just saying 'Hey if you have email for this domain, please send it.' For security reasons the client side cannot specify where to send the mail, just to send it now. (Now means add to queue. If the queue is busy it could take awhile for the email(s) to be received).

Notes: It does not hurt to issue ETRN commands if there is no mail held in the queue. No real error checking happens client side, it pretty much tries to connect & issue the commands blindly. Feel free to use telnet & issue the helo & ETRN manually for testing etc. Email must fail at least once before email is held for a secondary Route domain that is configured for ETRN. Once there is a failure ALL mail for that domain will instantly be held until a proper ETRN command is received. Upon ETRN, if delivery fails again mail will again be held. (This could happen if something is stopping delivery of email per the Route but ETRN's are still happening.)

GUI Status screen
=================
In the GUI under Status on Delivery queue tab clicking Refresh will show if any messages are held for a particular domain by the work 'Route HOLD' under Next try column.
Right-click on a message & choose Send now to force delivery try without ETRN.
Right-click in a message & choose Remove to delete it from the queue.
(You can select multiple messages in the list & choose either command but for obvious reasons USE CARE!)

HOW TO SETUP
==============

For ETRN Server side (where hmailserver holds/mailbags email for backup/secondary domains - NOT for primary domains hosted on the same box) you need to set that domain up as a Route Under Settings-Protocols-SMTP-Routes
- Enter in proper settings needed to route email traffic to a remote host that you are secondary for per http://www.hmailserver.com/documentatio ... _mx_backup
- IMPORTANT: Be sure to put ETRN as 1st 4 letters of the route description to enable mailbagging/holding for that domain. (The rest of the description is ignored & can be anything)

For ETRN client side (where hmailserver connects out & issues ETRN mydomain.com command to retrieve held emails) you need to setup a POP3 External account. (I realize that is odd but that was an easy place to get it into the GUI without tons of work)
- You'll need to create a domain & a user (doesn't matter if it'll be used or not. BE SURE TO HIT SAVE AFTER CREATING USER!) then on External accounts tab click Add.
- For name put ETRN yourdomain.com (replace yourdomain obviously and ETRN must be 1st 4 char of the string). NOTHING ELSE for the name!
- Leave type POP
- Enter IP or host name of the remote server you want to connect to & request release of mailbagged email. DO NOT enter your own IP or host name there!!
- Be sure to set port to proper SMTP port NOT POP port! (Odds are you want 25 as ETRN normally uses standard SMTP)
- Username/password should be left blank
- I have not tested with SSL but SSL isn't likely wanted/desired so leave unchecked
- Set Minutes between download to how often you want hmailserver to issue the ETRN command. DON'T DO TOO SHORT OF TIME! If unsure leave it at 30 min.
- Leave the rest of the settings at defaults as they should be ignored anyway.
- The 'Download now' button will force an ETRN if pressed but not required as the ETRN's will occur per the minutes set.

Hope ETRN functionality proves useful to people. I current use it for about 30 domains without problem. At some point I would guess time will be spent to catch the GUI up to the server with real ETRN settings areas but for how few people seem to want ETRN time is better spent on other things at the moment.
Bill

mgsuzen · Post by **mgsuzen** » 2014-01-29 09:47

I have a blacklist problem and i want to use ETRN mail some ISP's. But i can't configure my hmailserver. External accout tab is disable on my control panel. Please help me..

percepts · Post by **percepts** » 2014-01-29 13:42

You must post this type of question in the General section and NOT in the feature request section.

You must add and SAVE your user account BEFORE you can setup an external account fetch for it.

limengsiong · Post by **limengsiong** » 2015-05-29 10:57

hmailserver is a good email server product
But it lack of support for atrn , make it less than perfect app

At least it should support atrn client & server for hmailserver itself

hmailserver Sec (atrn server / mx backup) will hold all incoming emails while hmailserver Pri is offline.
When hmailserver Pri is online, it will sent atrn command to relay all emails from hmailserver Sec to hmailserver Pri.

Any timeline for this feature ?

Support for ETRN and ATRN [90%]

Do you need this feature?

Support for ETRN and ATRN [90%]

Re: Support for ETRN and ATRN

Re: Support for ETRN and ATRN

Re: Support for ETRN and ATRN

Re: Support for ETRN and ATRN

Re:

Re: Support for ETRN and ATRN

Re: Support for ETRN and ATRN

Re: Support for ETRN and ATRN

Re: Support for ETRN and ATRN

Re: Support for ETRN and ATRN

Re: Support for ETRN and ATRN

Re: Support for ETRN and ATRN

Re: Support for ETRN and ATRN

ETRN HOW-TO (as of 5.4 alpha)

Re: Support for ETRN and ATRN [90%]

Re: Support for ETRN and ATRN [90%]

Re: Support for ETRN and ATRN [90%]