Delete noreply still showing in queue

[cblaze22]

I setup a rule to delete anything going to noreply email address by testing if the message size > 0 and doing an action of delete on a rule on the account.

However, the queue still shows new items in it going to noreply? I would think they wouldnt hold up the queue in processing. We do about 1 million messages per day or more and have rejects which come back to our noreply and seem to stay there in the queue.

Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Return-Path: <>
Message-ID: <DB8C4919-4A88-4CDF-8378-45CE922934EA@email.test.com>
Date: Sat, 27 Apr 2019 23:16:34 -0400
From: mailer-daemon@email.test.com
To: noreply@test.com
Subject: Message undeliverable: 2019 Pittsburgh Jam Fest
Content-Transfer-Encoding: quoted-printable
X-hMailServer-LoopCount: 1

Your message did not reach some or all of the intended recipients.

Sent: 27 Apr 2019 22:05:20 -0400
Subject: 2019 Pittsburgh Jam Fest

The following recipient(s) could not be reached:

2152643050@comcastpcs.textmsg.com
Error Type: SMTP
Error Description: No mail servers appear to exists for the recipient's address.
Additional information: Please check that you have not misspelled the recipient's email address.


Tried 1 time(s)



hMailServer

[mattg]

Can you show your rule please
Is it an account level or global level rule?

[cblaze22]

It is on the account level otherwise it would delete all emails.

General
Name: Delete
Enabled

Criteria
Use And
Message Size Greater than 0

Actions
Delete e-mail

[mattg]

It is on the account level...

However, the queue still shows new items in it going to noreply?

SO the message must be delivered to the account, before the account rule can delete it...that seems correct to me.

The email error that you show has the recipient as unknown. In that instance the message can't be delivered to an account, and so the account rule can't fire...

If you want to drop all messages that are going to ANY no-reply address (not just those on your server) then you need a global rule with conditions to find these messages.

[cblaze22]

If I put the rule to drop messages to noreply@test.com on the global level, will it then drop it?

Keep in mind the message says To: noreply@test.com, which is an account within HMailServer, so it wont go anywhere anyway.

[mattg]

If I put the rule to drop messages to noreply@test.com on the global level, will it then drop it?

If you have delete message as an action of a global rule will that action happen? the answer is yes

[cblaze22]

Its still in my queue for processing. Does the rule not process until after it runs in the queue?

Code: Select all

Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Return-Path: <>
Message-ID: <9CFA04F2-937F-4DD2-8D7C-979B88E01B0F@test.com>
Date: Sat, 4 May 2019 18:43:00 -0400
From: mailer-daemon@test.com
To: noreply@test.com
Subject: Message undeliverable: Caveman Classic
Content-Transfer-Encoding: quoted-printable
X-hMailServer-LoopCount: 1

Your message did not reach some or all of the intended recipients.

   Sent: 4 May 2019 16:29:53 -0400
   Subject: Caveman Classic

The following recipient(s) could not be reached:

7243888948@txt.att.net
   Error Type: SMTP
   Remote server (166.216.152.132) issued an error.
   hMailServer sent: RCPT TO:<7243888948@txt.att.net>
   Remote server replied: 550 5.1.1 <7243888948@txt.att.net> recipient does not exist here.



hMailServer

[mattg]

Correct, for a rule to run on a message, hmailserver needs to process that message (by running it in the queue)

[cblaze22]

Well that sucks big time. I have like 1000 no reply messages just sitting in my queue because I cant delete them.

[mattg]

In the admin GUI you can 'clear queue' from the button at the top of the delivery queue page

[cblaze22]

That will clear all legit emails, like 39,000 of them.

[mattg]

You have 39 000 legitimate emails??

[cblaze22]

Yep....I run a tournament management service that sends game notifications so we have thousands going out a day

[mattg]

What are your delivery re-try settings?

Do you have the internet bandwidth to deliver tens of thousand of messages? Why are they building up in your queue?

[cblaze22]

0 retry. I mean I am on Azure with 2 cores and nothing else is used on this server except HMailServer. We just get so many notificadtions that come in I guess HMailServer cant keep up.

My Threading is

5
25
50
Highest

[mattg]

I'd set delivery threads to at least 100, possibly higher on that system
How much RAM is accessible on that system?
Can you get more cores? Two isn't very many for a machine that is that busy

How many messages per day?, I'm guessing many hundreds of thousands if you have 39K banked up

Asynchronous threads are used for AV and SpamAssassin and the like. Are you testing all messages for spam and AV?

[cblaze22]

I bumped it up to 100

7GB, currently only 3.5GB is being used.

I can get more cores and will probably do that before next weekend.

Probably over 100,000 per weekend day.

No need to test for Spam since we are sending the emails. Is there a way to turn that off so things go faster?

[mattg]

Run this and post the results
viewtopic.php?f=20&t=30914

[cblaze22]

Below are my results. I also donated to HMailServer a few days ago so we do appreciate you.

Code: Select all

2019-05-06   Hmailserver: 5.6.4-B2283

DOMAINS

   "Domain1.com" - emxxx.exxxxxxxxxxxxx.com       Enabled: True

SIGNATURE         LIMITS                       DKIM               ADVANCED
  Enabled: False   Max size:                0   Enabled: True    
                   Max message size:        0   Header:   Relaxed  Plus addressing: False
                   Max size of accounts:    0   Body:     Relaxed
                                                Algorithm: SHA256  Greylisting:     False
                                                Private key: c:\4573822996847382.Domain2.com.pem.txt
                                                Selector:    4573822996847382

   "Domain2.com" - exxxxxxxxxxxxx.com             Enabled: True

SIGNATURE         LIMITS                       DKIM               ADVANCED
  Enabled: False   Max size:                0   Enabled: True    
                   Max message size:        0   Header:   Relaxed  Plus addressing: False
                   Max size of accounts:    0   Body:     Relaxed
                                                Algorithm: SHA256  Greylisting:     False
                                                Private key: c:\4573822996847382.Domain2.com.pem.txt
                                                Selector:    4573822996847382
-----------------------------------------------------------------------------------------------

GLOBAL RULES
  1, Delete No Reply              Criteria:  Use AND
             To                        Equals          noreply@Domain2.com
                                  -----Actions-----
             Delete
-----------------------------------------------------------------------------------------------

IP RANGES

IP: 127.0.0.1 - 127.0.0.1     Priority: 50     Name: My computer

  Allow connections                         Other
     SMTP:   True                              Antispam :   True !! 'Spam tests' not enabled !!
     POP3:   True                              Antivirus:   True !! ANTIVIRUS NOT CONFIGURED !!
     IMAP:   True                              SSL/TLS:    False

  Allow Deliveries from                     Require Authentication from
     Local To Local       -  True              Local To Local       - False
     Local To External    -  True              Local To External    - False
     External To Local    -  True              External To Local    - False
     External To External -  True              External To External -  True


IP: 0.0.0.0 - 255.255.255.255     Priority: 10     Name: Internet

  Allow connections                         Other
     SMTP:   True                              Antispam :   True !! 'Spam tests' not enabled !!
     POP3:   True                              Antivirus:   True !! ANTIVIRUS NOT CONFIGURED !!
     IMAP:   True                              SSL/TLS:    False

  Allow Deliveries from                     Require Authentication from
     Local To Local       -  True              Local To Local       - False
     Local To External    -  True              Local To External    -  True
     External To Local    -  True              External To Local    - False
     External To External -  True              External To External -  True


   !!  Warning:  DEFAULT DOMAIN is SET  !! - "Domain1.com"
------------------------------------------------------
AUTOBANNED Local Addresses:
    No entries

-----------------------------------------------------------------------------------------------

AUTOBAN
  Autoban Enabled: True       Max invalid logon attempts:      3
                              Minutes Before Reset:           30  (0.50 hours, 0.02 days)
                              Minutes to Autoban:            600  (10.00 hours, 0.42 days)

No problems were found in the IP range configuration.
-----------------------------------------------------------------------------------------------

INCOMING RELAYS
   No entries
-----------------------------------------------------------------------------------------------

MIRRORING         Disabled
-----------------------------------------------------------------------------------------------

PROTOCOLS

SMTP
GENERAL             DELIVERY                  RFC COMPLIANCE            ADVANCED
No. Connections:  0  No Retries:  0 Mins:  0   Plain Text:        False  Bind: 
                     Host: Domain1.com         Empty sender:       True  Batch recipients:    25
Max Msg Size:200480  Relay:-                   Incorrect endings:  True  Use STARTTLS:      True
                     (none entered)            Disc. on invalid:  False  Delivered-To hdr: False
                                                                         Loop limit:           5
                                                                         Recipient hosts:     15
  Routes:
     No routes defined.

POP3
  No. Connections: 0

IMAP
 GENERAL                   PUBLIC FOLDERS                    ADVANCED
  No. Connections:   0      Public folder name: #Public       IMAP sort:  True
                                                              IMAP Quota: True
                                                              IMAP Idle:  True
                                                              IMAP ACL:   True
                                                              Delim: "."
-----------------------------------------------------------------------------------------------

ANTISPAM

GENERAL                              SPAM TESTS              Score   SPAMASSASSIN
  Spam Mark:                  20      Use SPF:           False        Use Spamassassin:   False
  Add X-HmailServer-Spam:     True    Check HELO host:   False    
  Add X-HmailServer-Reason:   True    Check MX records:  False    
  Add X-HmailServer-Subject:  True    Verify DKIM:       False    
              Subject Text: "[SPAM]"
  Spam delete threshold: 5         Maximum message size: 1024

DNSBL ENTRIES:
   No 'enabled' entries

SURBL ENTRIES:
   No 'enabled' entries

GREYLISTING:
  Greylisting:  False

WHITELISTING
   No entries
-----------------------------------------------------------------------------------------------

ANTIVIRUS:  No application configured.

  Block Attachments: False
-----------------------------------------------------------------------------------------------

SSL CERTIFICATES
   No entries
-----------------------------------------------------------------------------------------------

SSL/TLS
             SSL 3.0 :   True
             TLS 1.0 :   True
             TLS 1.1 :   True
             TLS 1.2 :   True                Verify Remote SSL/TLS Certs:   True
SslCipherList  :

ECDHE-RSA-AES128-GCM-SHA256     - ECDHE-ECDSA-AES128-GCM-SHA256   - ECDHE-RSA-AES256-GCM-SHA384     
ECDHE-ECDSA-AES256-GCM-SHA384   - DHE-RSA-AES128-GCM-SHA256       - DHE-DSS-AES128-GCM-SHA256       
kEDH+AESGCM                     - ECDHE-RSA-AES128-SHA256         - ECDHE-ECDSA-AES128-SHA256       
ECDHE-RSA-AES128-SHA            - ECDHE-ECDSA-AES128-SHA          - ECDHE-RSA-AES256-SHA384         
ECDHE-ECDSA-AES256-SHA384       - ECDHE-RSA-AES256-SHA            - ECDHE-ECDSA-AES256-SHA          
DHE-RSA-AES128-SHA256           - DHE-RSA-AES128-SHA              - DHE-DSS-AES128-SHA256           
DHE-RSA-AES256-SHA256           - DHE-DSS-AES256-SHA              - DHE-RSA-AES256-SHA              
AES128-GCM-SHA256               - AES256-GCM-SHA384               - ECDHE-RSA-RC4-SHA               
ECDHE-ECDSA-RC4-SHA             - AES128                          - AES256                          
RC4-SHA                         - HIGH                            - !aNULL                          
!eNULL                          - !EXPORT                         - !DES                            
!3DES                           - !MD5                            - !PSK;                           
-----------------------------------------------------------------------------------------------

TCPIP PORTS                                         Connection Sec
               0.0.0.0         / 110   / POP3   -   None                
               0.0.0.0         / 143   / IMAP   -   None                
               0.0.0.0         / 587   / SMTP   -   None                

    !! No SMTP Port 25 defined. Direct external SMTP inbound not possible !!

-----------------------------------------------------------------------------------------------

LOGGING      Logging Enabled: False

  Paths:-
    Error:    C:\Program Files (x86)\hMailServer\Logs\ERROR_hmailserver_2019-05-06.log
    Event:    C:\Program Files (x86)\hMailServer\Logs\hmailserver_events.log - Last Event: 2019/05/06
    Awstats:  C:\Program Files (x86)\hMailServer\Logs\hmailserver_awstats.log
-----------------------------------------------------------------------------------------------

SYSTEM TESTS

Database type: MSSQL

IPv6 support is available in operating system.

Backup directory C:\Users\cblaze22\Desktop is writable.

Relative message paths are stored in the database for all messages.

-----------------------------------------------------------------------------------------------

HMAILSERVER.INI

[Directories]
Program folder:  C:\Program Files (x86)\hMailServer\
Database folder: 
Data folder:     C:\Program Files (x86)\hMailServer\Data
Log folder:      C:\Program Files (x86)\hMailServer\Logs
Temp folder:     C:\Program Files (x86)\hMailServer\Temp
Event folder:    C:\Program Files (x86)\hMailServer\Events

[Database]
Type=              MSSQL
Username=          hmailserver
PasswordEncryption=1
Port=              0
Server=            127.0.0.1
Internal=          0
-----------------------------------------------------------------------------------------------

Generated by HMSSettingsDiagnostics v1.95, Hmailserver Forum.

[mattg]

So this machine doesn't accept any incoming mail, it only handles mail submitted via port 587. Correct?

what IP address(es) does the incoming mail get submitted from? I'm guessing this is from servers that you control.

Why do you have a default domain set?
Why do you have domains at all?

[cblaze22]

I control the servers that have permission to send to this server. It only sends out email via SMTP.

I use the domains for any loop issues, so emails will go to an external email by taking advantage of the forwarding.

[mattg]

I use the domains for any loop issues, so emails will go to an external email by taking advantage of the forwarding.

Please give more detail about this concept...what are you hoping to achieve...and how

[cblaze22]

If I send an email to say info@test.com from noreply@test.com, it forwards to my gmail account. We do this so we can test with our own domain otherwise it stays in a loop in HMailServer and never makes it through.

Plus dont we need domains so we can use the DKIM?

[mattg]

Yes you do need a domain to use DKIM

By forwarding you are creating an extra message, effectively doubling the load. hMailserver has a loop count, so that messages aren't looped forever. This is a safety feature to stop the queue filling with dead messages

Why do you have a default domain set? Is this also for DKIM purposes?

[cblaze22]

The doubling isnt a big deal in that case since we send to ourselves rarely. If I want noreply@test.com to go to info@test.com, I have to have a valid email to get out of the loop like test@gmail.com. I know there is a loop count, but the point is I need to get the email regardless, forwarding takes care of this, unless you know of a better way.

Where is the default domain at and why is that a problem?

[mattg]

Settings >> Advanced

having a default domain MAY create some bad DKIM signing, because hMailserver assumes that a FROM of 'coyote' is in fact FROM 'coyote@example.com' if example.com is the default domain.

[cblaze22]

Ill take your word for it. I removed it. Anything else you see needed in my report?

[mattg]

Only that I'd turn SSLv3.0 (at least) off - It is known to be broken

[cblaze22]

Will do, I also increased from 2 to 4 cores this weekend to see how the queue goes.

[cblaze22]

Good weekend, nothing in log at all. I think the cores helped. However I still get the original question issues where the email go into the queue for no reason. If I set some code in OnAcceptMessage to test for From: mailer-daemon@email.test.com I believe I can drop it there correct and not have it go into the queue.

[mattg]

I reject messages in OnAcceptMessage but I don't delete them
Result.Value = 2


Unsure if delete will work or not

[cblaze22]

So if I test for the mail-demon it will drop them if I do that in OnAcceptQueue and they wont show up in the queue?

[SorenR]

oMessages.DeleteByDBID is not an option until the message have arrived in the mailbox - after completing rule handling and the EventHandlers script. The alternative is to "do your stuff" and then send the email to a wastebasket or tag it with a header value and delete it by rule.

I did a test at the different levels to see what information was in the hm_messages table. Within the span of the EventHandlers script no AccountID not FolderID was observed thus no owner of the message was found. The implementation of the Messages object require either account object or account + folder object to be known.

Result.Value = 2 (or 1) will send an NDR with or without an explanation. You probably do not want to reply to a NDR with another NDR ...