DKIM on reply from rule

Use this forum if you have installed hMailServer and want to ask a question related to a production release of hMailServer. Before posting, please read the troubleshooting guide. A large part of all reported issues are already described in detail here.
Post Reply
User avatar
weinberk
Normal user
Normal user
Posts: 119
Joined: 2009-01-24 00:17

DKIM on reply from rule

Post by weinberk » 2019-05-08 05:16

I've DKIM setup on a subdomain without issue. We have a handful of accounts that send an automatic reply via a reply RULE (not an auto-reply from the auto-reply tab) so that senders get a reply for every single email they send to a given mailbox.

This works, but the messages don't appear to be DKIM signed. Could this be a bug? Other messages from this subdomain sent via SMTP connections do get signed and validate.

Thanks

User avatar
RvdH
Senior user
Senior user
Posts: 3235
Joined: 2008-06-27 14:42
Location: The Netherlands

Re: DKIM on reply from rule

Post by RvdH » 2019-05-08 10:27

CIDR to RegEx: d-fault.nl/cidrtoregex
DNS Lookup: d-fault.nl/dnstools
DKIM Generator: d-fault.nl/dkimgenerator
DNSBL Lookup: d-fault.nl/dnsbllookup
GEOIP Lookup: d-fault.nl/geoiplookup

User avatar
RvdH
Senior user
Senior user
Posts: 3235
Joined: 2008-06-27 14:42
Location: The Netherlands

Re: DKIM on reply from rule

Post by RvdH » 2019-05-08 13:34

The problem with such rule is that the filled in email address in Reply rule doesn't have to be a local address/domain. You can type anything you like there.
That, as well as the missing SMTPFromAddress (empty due to Auto-Submitted header) is responsible for not signing the message (i think ;) )
CIDR to RegEx: d-fault.nl/cidrtoregex
DNS Lookup: d-fault.nl/dnstools
DKIM Generator: d-fault.nl/dkimgenerator
DNSBL Lookup: d-fault.nl/dnsbllookup
GEOIP Lookup: d-fault.nl/geoiplookup

User avatar
weinberk
Normal user
Normal user
Posts: 119
Joined: 2009-01-24 00:17

Re: DKIM on reply from rule

Post by weinberk » 2019-05-08 16:02

(Embarrassed that I didn't come across the known bug via search)

I guess I'll try using a script for the rule based reply. Not optimal, but apparently the only way to quash this 4+ year old known bug / feature limitation.

Many email providers, including gmail, are enforcing DMARC these days. That's a great thing as it helps to prevent our own domain from being spoofed. But if DKIM is required (dmarc of p=reject or p=quarantine), hmail is setup to DKIM sign, but hmail doesn't always sign matching domains, that's a problem...

User avatar
RvdH
Senior user
Senior user
Posts: 3235
Joined: 2008-06-27 14:42
Location: The Netherlands

Re: DKIM on reply from rule

Post by RvdH » 2019-05-09 10:48

I think this Reply rule is kinda weird anyway....why would one allow to Reply from another account other then the account this rule is applied to in the first place???

@weinberk, how are your coding skills? Changes below DKIM Sign messages using account-level reply rules

RuleApplier.h
replace:

Code: Select all

void ApplyAction_Reply(std::shared_ptr<RuleAction> pAction, std::shared_ptr<MessageData> pMsgData) const;
with:

Code: Select all

void ApplyAction_Reply(std::shared_ptr<RuleAction> pAction, std::shared_ptr<const Account> account, std::shared_ptr<MessageData> pMsgData) const;
RuleApplier.cpp

replace:

Code: Select all

      case RuleAction::Reply:
         {
            ApplyAction_Reply(pAction, pMsgData);
            break;
         }
with:

Code: Select all

      case RuleAction::Reply:
         {
	    ApplyAction_Reply(pAction, account, pMsgData);
            break;
         }
replace the whole:

Code: Select all

   void 
   RuleApplier::ApplyAction_Reply(std::shared_ptr<RuleAction> pAction, std::shared_ptr<MessageData> pMsgData) const
   {
   ...
   }
with:

Code: Select all

   void 
   RuleApplier::ApplyAction_Reply(std::shared_ptr<RuleAction> pAction, std::shared_ptr<const Account> account, std::shared_ptr<MessageData> pMsgData) const
   {
      // true = check AutoSubmitted header and do not respond if set
      if (!IsGeneratedResponseAllowed(pMsgData, true))
      {
         ErrorManager::Instance()->ReportError(ErrorManager::Medium, 5065, "RuleApplier::ApplyAction_Reply", "Could not reply message. Maximum rule loop count reached or Auto-Submitted header.");
         return;
      }

      String sReplyRecipientAddress  = pMsgData->GetMessage()->GetFromAddress();

      if (sReplyRecipientAddress.IsEmpty())
      {
         // We need a recipient address to be able to
         // send the message..
         return;
      }

      std::shared_ptr<Account> emptyAccount;

      // Sen d a copy of this email.
      std::shared_ptr<Message> pMsg = std::shared_ptr<Message>(new Message());
      pMsg->SetState(Message::Delivering);
      
      String newMessageFileName = PersistentMessage::GetFileName(pMsg);

      std::shared_ptr<MessageData> pNewMsgData = std::shared_ptr<MessageData>(new MessageData());
      pNewMsgData->LoadFromMessage(newMessageFileName, pMsg);
      pNewMsgData->SetReturnPath("");
      pNewMsgData->GenerateMessageID();
      pNewMsgData->SetTo(sReplyRecipientAddress);
      pNewMsgData->SetFrom(pAction->GetFromName() + " <" + pAction->GetFromAddress() + ">");
      pNewMsgData->SetSubject(pAction->GetSubject());
      pNewMsgData->SetBody(pAction->GetBody());
      pNewMsgData->SetSentTime(Time::GetCurrentMimeDate());
      pNewMsgData->SetAutoReplied();
	  pNewMsgData->IncreaseRuleLoopCount();
      pNewMsgData->Write(newMessageFileName);

	  // We need to update the SMTP envelope from address, if this
	  // message is replied to by a user-level account.
	  std::shared_ptr<CONST Account> pAccount = CacheContainer::Instance()->GetAccount(rule_account_id_);
	  if (pAccount)
		  pMsg->SetFromAddress(pAccount->GetAddress());

      // Add recipients.
      bool recipientOK = false;
      RecipientParser recipientParser;
      recipientParser.CreateMessageRecipientList(sReplyRecipientAddress, pMsg->GetRecipients(), recipientOK);

      PersistentMessage::SaveObject(pMsg);

   }
CIDR to RegEx: d-fault.nl/cidrtoregex
DNS Lookup: d-fault.nl/dnstools
DKIM Generator: d-fault.nl/dkimgenerator
DNSBL Lookup: d-fault.nl/dnsbllookup
GEOIP Lookup: d-fault.nl/geoiplookup

User avatar
RvdH
Senior user
Senior user
Posts: 3235
Joined: 2008-06-27 14:42
Location: The Netherlands

Re: DKIM on reply from rule

Post by RvdH » 2019-05-09 14:25

Fix... added check if it is a global rule or an account rule
global rule = not DKIM signed, Empty return-path header (this is the current behaviour!)
account rule = DKIM signed (if enabled), Non empty return-path header

Replace the whole:

Code: Select all

   void 
   RuleApplier::ApplyAction_Reply(std::shared_ptr<RuleAction> pAction, std::shared_ptr<MessageData> pMsgData) const
   {
   ...
   }

Code: Select all

   void 
   RuleApplier::ApplyAction_Reply(std::shared_ptr<RuleAction> pAction, std::shared_ptr<const Account> account, std::shared_ptr<MessageData> pMsgData) const
   {
      // true = check AutoSubmitted header and do not respond if set
      if (!IsGeneratedResponseAllowed(pMsgData, true))
      {
         ErrorManager::Instance()->ReportError(ErrorManager::Medium, 5065, "RuleApplier::ApplyAction_Reply", "Could not reply message. Maximum rule loop count reached or Auto-Submitted header.");
         return;
      }

      String sReplyRecipientAddress  = pMsgData->GetMessage()->GetFromAddress();

      if (sReplyRecipientAddress.IsEmpty())
      {
         // We need a recipient address to be able to
         // send the message..
         return;
      }

      std::shared_ptr<Account> emptyAccount;

      // Send a copy of this email.
      std::shared_ptr<Message> pMsg = std::shared_ptr<Message>(new Message());
      pMsg->SetState(Message::Delivering);
      
      String newMessageFileName = PersistentMessage::GetFileName(pMsg);

	  // check if this us a user-level account rule or global rule.
	  std::shared_ptr<CONST Account> pAccount = CacheContainer::Instance()->GetAccount(rule_account_id_);

      std::shared_ptr<MessageData> pNewMsgData = std::shared_ptr<MessageData>(new MessageData());
      pNewMsgData->LoadFromMessage(newMessageFileName, pMsg);
	  if (!pAccount)
		  pNewMsgData->SetReturnPath("");
      pNewMsgData->GenerateMessageID();
      pNewMsgData->SetTo(sReplyRecipientAddress);
      pNewMsgData->SetFrom(pAction->GetFromName() + " <" + pAction->GetFromAddress() + ">");
      pNewMsgData->SetSubject(pAction->GetSubject());
      pNewMsgData->SetBody(pAction->GetBody());
      pNewMsgData->SetSentTime(Time::GetCurrentMimeDate());
      pNewMsgData->SetAutoReplied();
	  pNewMsgData->IncreaseRuleLoopCount();
      pNewMsgData->Write(newMessageFileName);

	  // We need to update the SMTP envelope from address, if this
	  // message is replied to by a user-level account.
	  if (pAccount)
		  pMsg->SetFromAddress(pAccount->GetAddress());

      // Add recipients.
      bool recipientOK = false;
      RecipientParser recipientParser;
      recipientParser.CreateMessageRecipientList(sReplyRecipientAddress, pMsg->GetRecipients(), recipientOK);

      PersistentMessage::SaveObject(pMsg);

   }
CIDR to RegEx: d-fault.nl/cidrtoregex
DNS Lookup: d-fault.nl/dnstools
DKIM Generator: d-fault.nl/dkimgenerator
DNSBL Lookup: d-fault.nl/dnsbllookup
GEOIP Lookup: d-fault.nl/geoiplookup

User avatar
RvdH
Senior user
Senior user
Posts: 3235
Joined: 2008-06-27 14:42
Location: The Netherlands

Re: DKIM on reply from rule

Post by RvdH » 2019-06-02 09:38

If you would like to give my build with this functionality a go, it is found here: https://www.hmailserver.com/forum/viewt ... 68#p212268
CIDR to RegEx: d-fault.nl/cidrtoregex
DNS Lookup: d-fault.nl/dnstools
DKIM Generator: d-fault.nl/dkimgenerator
DNSBL Lookup: d-fault.nl/dnsbllookup
GEOIP Lookup: d-fault.nl/geoiplookup

User avatar
weinberk
Normal user
Normal user
Posts: 119
Joined: 2009-01-24 00:17

Re: DKIM on reply from rule

Post by weinberk » 2020-05-01 21:01

Heh, I must have missed your reply notification RvdH! So here I am, almost a year later, back at this issue again.
There seems to have been a little development work recently (mostly OpenSSL updates), but do you know if Martin integrated any of your changes in to 5.6.x?

Thanks!

User avatar
RvdH
Senior user
Senior user
Posts: 3235
Joined: 2008-06-27 14:42
Location: The Netherlands

Re: DKIM on reply from rule

Post by RvdH » 2020-05-01 21:49

No, none of these are in any of the official 5.6.x builds, but most of the changes from my custom build made it to the latest 5.7.0 version, although it is advised to not use in production just yet
CIDR to RegEx: d-fault.nl/cidrtoregex
DNS Lookup: d-fault.nl/dnstools
DKIM Generator: d-fault.nl/dkimgenerator
DNSBL Lookup: d-fault.nl/dnsbllookup
GEOIP Lookup: d-fault.nl/geoiplookup

User avatar
weinberk
Normal user
Normal user
Posts: 119
Joined: 2009-01-24 00:17

Re: DKIM on reply from rule

Post by weinberk » 2020-05-01 23:29

Thanks for the quick reply. We'll have to continue dealing with script replies not being DKIM signed. Messes with strict DMARC, but owell.

...breath held that a stable 5.7 release sees the light of day soon.

thanks for everything

User avatar
RvdH
Senior user
Senior user
Posts: 3235
Joined: 2008-06-27 14:42
Location: The Netherlands

Re: DKIM on reply from rule

Post by RvdH » 2020-05-02 00:07

Or use my custom 5.6.x build provided till that time? Latest is 5.6.8-B2505.26
Just install the latest production and/or beta version from the URL below, then copy and overwrite files in this archive in hmailserver '/bin' directory
https://build.hmailserver.com/viewLog.h ... =artifacts (login as guest)
  1. Supports Sub OnHELO(oClient) event, issue #153
  2. Fixed Incorrect DEBUG logging for event 'OnDeliverMessage', issue #181
  3. Include HTMLBody into IMAP TEXT search, pull #193
  4. Fixed implicit conversion: "int" to "unsigned char" pull #204
  5. Faulty: SMTP 'Disconnect client after too many invalid commands' pull issue #160
  6. SMTP server error "550 Unsupported ESMTP extension" on MAIL FROM:... AUTH=<> [with fix] issue #164
  7. Removed warning if backup was more than 1,5GB and 15GB limit. There's no longer a recommended max-size - the time will vary with the installation size. issue #69
  8. Speed up 'update hm_messages set messageflags' issue #221
  9. Treat authenticated users as localsender if the sender is authenticated and AuthUserIsLocal=1 INI setting Office 2016/2019 Bug
  10. Add Return-Path header as topmost header before sending the message to SA (+ delete Return-Path header after the SA check completes) issue #116
  11. Experimental eventhandler OnClientLogon(oClient), New ClientInfo property oClient.Authenticated (Boolean)
  12. Handling of long UIDL response lists was too slow. issue #93
  13. When calling SpamAssassin and there was a connection failure, sometimes temporary files were left behind issue #100
  14. SURBL detection properly fails to detect url's ending with a query string issue #108
  15. If a route is set up, but the recipient does not match an address in the route address list, the domain catch-all should be used if specified. issue #74
  16. Fix ExternalFetcher DELE when no RETR, pull pull #254
  17. SMTP multiply max message size with 1024 issue #267
  18. Add email address variable to SignatureAdder.cpp pull #265
  19. DKIM on acccount-rule 'reply' not applied #172 issue #172
  20. preserve RewriteEnvelopeFromWhenForwarding setting when forwarding from account rule
  21. The logical flow should be to disregard "Require SMTP authentication" if "Allow deliveries from" is unselected issue #287
  22. Add ability to DKIM sign NDR messages (forwarded to external) pull #301
  23. Use custom daemonaddressdomain from INI pull #301
  24. Fix SURBL regex pull #320
  25. Add RMSPF library SpamTestSPF Result to DEBUG logging, see this forum topic
  26. Ignore SpamTestSPF and SpamTestHeloHost when send thru local IP Address, see this forum topic
https://d-fault.nl/files/hMailServer-Bu ... 2505.26.7z
CIDR to RegEx: d-fault.nl/cidrtoregex
DNS Lookup: d-fault.nl/dnstools
DKIM Generator: d-fault.nl/dkimgenerator
DNSBL Lookup: d-fault.nl/dnsbllookup
GEOIP Lookup: d-fault.nl/geoiplookup

User avatar
weinberk
Normal user
Normal user
Posts: 119
Joined: 2009-01-24 00:17

Re: DKIM on reply from rule

Post by weinberk » 2020-05-02 00:17

I assumed your build was outdated, but now I see that the latest was built on 4/24. G-R-E-A-T
Stability is as good as the previous official release? Any known issues?

User avatar
RvdH
Senior user
Senior user
Posts: 3235
Joined: 2008-06-27 14:42
Location: The Netherlands

Re: DKIM on reply from rule

Post by RvdH » 2020-05-02 00:24

Basically it is as stable as all other 5.6.8 builds... 5.6.8-B2505 is nothing more then the most recent OpenSSL update only with the above list of fixes and tweaks incorporated of with no known issues, quite a few here are using those custom builds without complains
CIDR to RegEx: d-fault.nl/cidrtoregex
DNS Lookup: d-fault.nl/dnstools
DKIM Generator: d-fault.nl/dkimgenerator
DNSBL Lookup: d-fault.nl/dnsbllookup
GEOIP Lookup: d-fault.nl/geoiplookup

User avatar
weinberk
Normal user
Normal user
Posts: 119
Joined: 2009-01-24 00:17

Re: DKIM on reply from rule

Post by weinberk » 2020-05-02 00:28

Outstanding. I'll give it a go likely next week. I appreciate all you've done for this community (And me).

User avatar
weinberk
Normal user
Normal user
Posts: 119
Joined: 2009-01-24 00:17

Re: DKIM on reply from rule

Post by weinberk » 2020-05-02 19:02

@RvdH,

I have it a go. 5.6.8-B2505 installed over my existing installation without issue. I can see in the admin panel that this version is running.

I've confirmed that a regular autoreply for that same account does sign just fine through hmailserver. However, a rule with a reply, using the same from, with the same case even, doesn't have a dkim signature at all on the reply. It's not that it's being signed incorrectly, there's no signature at all.

Is there something I need to do in a config file or something to specifically enable dkim signing on reply rules?

User avatar
RvdH
Senior user
Senior user
Posts: 3235
Joined: 2008-06-27 14:42
Location: The Netherlands

Re: DKIM on reply from rule

Post by RvdH » 2020-05-02 21:31

No nothing special to configure, please note that only messages form local address to external address are DKIM signed
I guess you are testing from local to another local address? And this is a account-rule, right? On a global-rule it won't work (no account-id accessible in that circumstance)
CIDR to RegEx: d-fault.nl/cidrtoregex
DNS Lookup: d-fault.nl/dnstools
DKIM Generator: d-fault.nl/dkimgenerator
DNSBL Lookup: d-fault.nl/dnsbllookup
GEOIP Lookup: d-fault.nl/geoiplookup

User avatar
weinberk
Normal user
Normal user
Posts: 119
Joined: 2009-01-24 00:17

Re: DKIM on reply from rule

Post by weinberk » 2020-05-03 05:29

Yep. Rule in user@domain.com. reply rule sending as that same user@domain.com. regular auto reply is signed.

User avatar
RvdH
Senior user
Senior user
Posts: 3235
Joined: 2008-06-27 14:42
Location: The Netherlands

Re: DKIM on reply from rule

Post by RvdH » 2020-05-03 10:53

weinberk wrote:
2020-05-03 05:29
Yep. Rule in user@domain.com. reply rule sending as that same user@domain.com. regular auto reply is signed.
No issue here, you did overwrite the files in /Bin with the *.7z package after installation of the latest 5.8.x build???
What does the version read in hMailServer Administrator?

The fact the rule is sending as that same user is irrelevant as it sets the EnvelopeFrom address using the account id attached to the rule, eg: that is reason it only works to account-rules

relevant code

Code: Select all

	  // check if this us a user-level account rule or global rule.
	  std::shared_ptr<CONST Account> pAccount = CacheContainer::Instance()->GetAccount(rule_account_id_);
	  
	  if (!pAccount)
		  pNewMsgData->SetReturnPath("");

	  // We need to update the SMTP envelope from address, if this
	  // message is replied to by a user-level account.
	  if (pAccount)
		  pMsg->SetFromAddress(pAccount->GetAddress());
CIDR to RegEx: d-fault.nl/cidrtoregex
DNS Lookup: d-fault.nl/dnstools
DKIM Generator: d-fault.nl/dkimgenerator
DNSBL Lookup: d-fault.nl/dnsbllookup
GEOIP Lookup: d-fault.nl/geoiplookup

User avatar
weinberk
Normal user
Normal user
Posts: 119
Joined: 2009-01-24 00:17

Re: DKIM on reply from rule

Post by weinberk » 2020-05-03 17:31

IT'S WORKING. Outstanding. I had inadvertently extracted the 7z into a subfolder of /bin.
Terrific work.

Post Reply