Account rule deletes message but not the reference

Use this forum if you have installed hMailServer and want to ask a question related to a production release of hMailServer. Before posting, please read the troubleshooting guide. A large part of all reported issues are already described in detail here.
Post Reply
User avatar
katip
Senior user
Senior user
Posts: 629
Joined: 2006-12-22 07:58
Location: Istanbul

Account rule deletes message but not the reference

Post by katip » 2019-04-12 11:25

account rule #1:

if message size > 0
run a function

function does some checkings, adds an header according to the result, saves the message and returns.

then comes account rule #2:
if x-header (which was added by above function) contains "Yes"
and loopcount < 1
forward to xyz@domain.com
delete message
stop rule processing (with and without, tried both)

=no more rule=

works perfect. BUT user (rule's user) gets a postmaster message after some minutes when downloading his/her messages by POP:
"The mail server could not deliver the message to you since the file C:\hMailServer\Data\domain.com\user\0B\{0BB87CB6-C481-4CA4-A483-4AD73CC40F31}.eml does not exist on the server."

file is gone as desired, but what happens with the reference claiming that file?
a simple account rule like "if message size > x delete it" doesn't produce this error. why this?
Katip
--
HMS 5.7.0-B2428-LTS-64-bit, MySQL 5.7.24, SA 3.4.2, ClamAV 0.101.2 + SaneS

User avatar
SorenR
Senior user
Senior user
Posts: 3063
Joined: 2006-08-21 15:38
Location: Denmark

Re: Account rule deletes message but not the reference

Post by SorenR » 2019-04-12 15:08

Both rules are "account rules"?

A while back I spent some time tracking (debug and application log) when rules are applied as a pre-study to my SPAM handling. I ended up loading everything into "global rules".
I found that actions like "Forward" and "Move to IMAP.." was actually performed at "account rule" level although it was defined in a "global rule".

So, I now have 2 (global) rules handling SPAM, first one check "X-hMailServer-Reason-Score" and if > 6 it is forwarded to "spam@acme.inc", mail is deleted and rule stop processing. Second one just forward to "spam@acme.inc" and move mail to IMAP folder.

Never had any issues about missing emails from postmaster.

Could it be something from the script that gets stuck in memory? I know VBScript is supposed to "clean up" when exiting but do you "nothing" all your objects when you are done with them?
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
katip
Senior user
Senior user
Posts: 629
Joined: 2006-12-22 07:58
Location: Istanbul

Re: Account rule deletes message but not the reference

Post by katip » 2019-04-12 16:16

both rules are from same account.

you're right with global rules. i use also exclusively global rules to sort spam and change headers etc...
till last week i was using globals also for redirectings. due to multiple receipts by splitted mails issue i told about before, i changed this policy and dropped all redirectings by global rules and started to proceed with indexing, querying etc.. by functions called from account rules & evaluate the result again in account rules. you know the story. this runs perfectly.

now after that bad experience with public folders i told about yesterday, now i'm trying to forward discarded messages to a trash account & delete on account rule level. this runs also fine, but unexpectedly leaves behind this postmaster warning. users are upset and start asking what is happening. (according to my current observations we have about 500-600 discards daily)

ok, i'll try to cleanup functions with Nothings etc.. we'll see.
in worst case i'll go further blind with public folder and follow discards from daily event logs (Timestamp/Action/From/To/Subject). at least users keep silent. better than nothing..
Katip
--
HMS 5.7.0-B2428-LTS-64-bit, MySQL 5.7.24, SA 3.4.2, ClamAV 0.101.2 + SaneS

User avatar
SorenR
Senior user
Senior user
Posts: 3063
Joined: 2006-08-21 15:38
Location: Denmark

Re: Account rule deletes message but not the reference

Post by SorenR » 2019-04-12 17:02

katip wrote:
2019-04-12 16:16
both rules are from same account.

you're right with global rules. i use also exclusively global rules to sort spam and change headers etc...
till last week i was using globals also for redirectings. due to multiple receipts by splitted mails issue i told about before, i changed this policy and dropped all redirectings by global rules and started to proceed with indexing, querying etc.. by functions called from account rules & evaluate the result again in account rules. you know the story. this runs perfectly.

now after that bad experience with public folders i told about yesterday, now i'm trying to forward discarded messages to a trash account & delete on account rule level. this runs also fine, but unexpectedly leaves behind this postmaster warning. users are upset and start asking what is happening. (according to my current observations we have about 500-600 discards daily)

ok, i'll try to cleanup functions with Nothings etc.. we'll see.
in worst case i'll go further blind with public folder and follow discards from daily event logs (Timestamp/Action/From/To/Subject). at least users keep silent. better than nothing..
Most of the time it is some silly little thing that tip the big load. You can look at it for days and then suddenly you see it :mrgreen:
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
katip
Senior user
Senior user
Posts: 629
Joined: 2006-12-22 07:58
Location: Istanbul

Re: Account rule deletes message but not the reference

Post by katip » 2019-04-13 20:05

i did some debug as seen below. scenario:
i send one mail to one external address + CC a distribution list where i'm also in it.
aim : discard the copy from distrolist to myself, forward it to trash account to be reviewed
once again, account rules in my account :
account rule #1:
if message size > 0
run a function
(function does some checkings, adds an header according to the result, saves the message and returns.)
account rule #2:
if x-header (which was added by above function) contains "Yes"
forward to trash@mydomain.com
delete message

=no more rule=

debug (*** comments are my worthless assumptions) :

Code: Select all

"SMTPD"	4068	25021	"2019-04-13 19:34:41.463"	"192.168.1.200"	"RECEIVED: MAIL FROM:<myself@mydomain.com>"
"SMTPD"	4068	25021	"2019-04-13 19:34:41.463"	"192.168.1.200"	"SENT: 250 OK"
"SMTPD"	4700	25021	"2019-04-13 19:34:41.479"	"192.168.1.200"	"RECEIVED: RCPT TO:<testlist@mydomain.com>"
"DEBUG"	4700	"2019-04-13 19:34:41.494"	"DistributionList::LMPublic"
"SMTPD"	4700	25021	"2019-04-13 19:34:41.494"	"192.168.1.200"	"SENT: 250 OK"
"SMTPD"	1812	25021	"2019-04-13 19:34:41.494"	"192.168.1.200"	"RECEIVED: DATA"
"SMTPD"	1812	25021	"2019-04-13 19:34:41.494"	"192.168.1.200"	"SENT: 354 OK, send."
"DEBUG"	1812	"2019-04-13 19:34:41.494"	"Adding task AsynchronousTask to work queue Asynchronous task queue"
"DEBUG"	2516	"2019-04-13 19:34:41.510"	"Executing task AsynchronousTask in work queue Asynchronous task queue"
"DEBUG"	2516	"2019-04-13 19:34:41.510"	"Executing event OnAcceptMessage"
"DEBUG"	2516	"2019-04-13 19:34:41.791"	"Event completed"
"DEBUG"	2516	"2019-04-13 19:34:41.791"	"Saving message: {FAD405C8-908B-40CF-9BF0-E43ABC1C20E9}.eml"
"DEBUG"	2516	"2019-04-13 19:34:42.041"	"Requesting SMTPDeliveryManager to start message delivery"
"SMTPD"	2516	25021	"2019-04-13 19:34:42.041"	"192.168.1.200"	"SENT: 250 Queued (0.000 seconds)"
"SMTPD"	1812	25021	"2019-04-13 19:34:42.057"	"192.168.1.200"	"RECEIVED: QUIT"
"SMTPD"	1812	25021	"2019-04-13 19:34:42.057"	"192.168.1.200"	"SENT: 221 goodbye"
"DEBUG"	4700	"2019-04-13 19:34:42.057"	"Ending session 25021"
"DEBUG"	3852	"2019-04-13 19:34:42.104"	"Adding task DeliveryTask to work queue SMTP delivery queue"
"DEBUG"	4344	"2019-04-13 19:34:42.104"	"Executing task DeliveryTask in work queue SMTP delivery queue"
"DEBUG"	4344	"2019-04-13 19:34:42.104"	"Delivering message..."
"APPLICATION"	4344	"2019-04-13 19:34:42.104"	"SMTPDeliverer - Message 369695: Delivering message from myself@mydomain.com to myself@mydomain.com, someone@external-domain.com. File: C:\hMailServer\Data\{FAD405C8-908B-40CF-9BF0-E43ABC1C20E9}.eml" *** <- list members ***
"DEBUG"	4344	"2019-04-13 19:34:42.104"	"Applying rules"
(... some irrelevant global rules ...)
"DEBUG"	4344	"2019-04-13 19:34:42.104"	"Performing local delivery"
"DEBUG"	4344	"2019-04-13 19:34:42.104"	"Copying mail contents"
"DEBUG"	4344	"2019-04-13 19:34:42.119"	"Applying rules" *** <- account rules start ***
"DEBUG"	4344	"2019-04-13 19:34:42.119"	"Applying rule check_if_dup" *** <- this is where function is called ***
"DEBUG"	4344	"2019-04-13 19:34:42.119"	"Performing rule action"
"DEBUG"	4344	"2019-04-13 19:34:42.119"	"Executing event Unknown"
"DEBUG"	4344	"2019-04-13 19:34:42.166"	"Saving message: {9C49A54E-798C-48EA-B8FC-B86F55CB799B}.eml" *** <- this is the new file after oMessage.Save to save header***
"DEBUG"	4068	"2019-04-13 19:34:42.244"	"Deleting message" *** <- initial file {FAD405C8-908B-40CF-9BF0-E43ABC1C20E9}.eml is deleted ***
"DEBUG"	3372	"2019-04-13 19:34:42.244"	"No messages to index."
"DEBUG"	4068	"2019-04-13 19:34:42.432"	"Deleting message file."
"DEBUG"	4344	"2019-04-13 19:34:42.541"	"Event completed" *** <- return from function ***
"DEBUG"	4344	"2019-04-13 19:34:42.541"	"Applying rule kill_dup" *** <- this is where rule looks for header value and deletes the message ***
"DEBUG"	4344	"2019-04-13 19:34:42.541"	"Performing rule action"
"DEBUG"	4344	"2019-04-13 19:34:42.541"	"Copying mail contents" *** <- this is for forwarding to trash account ***
"DEBUG"	4344	"2019-04-13 19:34:42.557"	"Saving message: {1B04D6E5-B3C0-49C6-8545-30ACBF41389D}.eml" *** <- to be sent to trash account ***
"DEBUG"	4344	"2019-04-13 19:34:42.838"	"Performing rule action" *** <- delete {9C49A54E-798C-48EA-B8FC-B86F55CB799B}.eml ??? ***
"APPLICATION"	4344	"2019-04-13 19:34:42.838"	"SMTPDeliverer - Message 369696: The message was not delivered to myself@mydomain.com. Delivery to this account was canceled by an account rule Rule name: kill_dup, ID: 20."
"DEBUG"	4344	"2019-04-13 19:34:42.885"	"Local delivery completed"
(...)
{9C49A54E-798C-48EA-B8FC-B86F55CB799B}.eml is gone but it remains in hm_messages.
when retrieving POP or IMAP, HMS creates a Postmaster warning with the same file name and user is freaked out :P

Code: Select all

From: Postmaster
Subject: Message undeliverable
Date: Sat, 13 Apr 2019 19:45:20 +0300

The mail server could not deliver the message to you since the file C:\hMailServer\Data\mydomain.com\myself\9C\{9C49A54E-798C-48EA-B8FC-B86F55CB799B}.eml does not exist on the server.

The file may have been deleted by anti virus software running on the server.

hMailServer
in short : don't touch the message's source on account rule level and then try to delete it from there.
Katip
--
HMS 5.7.0-B2428-LTS-64-bit, MySQL 5.7.24, SA 3.4.2, ClamAV 0.101.2 + SaneS

User avatar
katip
Senior user
Senior user
Posts: 629
Joined: 2006-12-22 07:58
Location: Istanbul

Re: Account rule deletes message but not the reference

Post by katip » 2019-04-13 20:22

Untitled 1.png
Untitled 1.png (8.1 KiB) Viewed 587 times
Katip
--
HMS 5.7.0-B2428-LTS-64-bit, MySQL 5.7.24, SA 3.4.2, ClamAV 0.101.2 + SaneS

User avatar
SorenR
Senior user
Senior user
Posts: 3063
Joined: 2006-08-21 15:38
Location: Denmark

Re: Account rule deletes message but not the reference

Post by SorenR » 2019-04-13 22:26

I just tried to replicate your results... My logs are similar to yours but I don't get the message from Postmaster, maybe because I use IMAP.

I added the rules as global rules and the log is slightly different, there is no "DeliveryTask" and there is no ""SMTPDeliverer - Message 369696: The message was not delivered to m......"
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
katip
Senior user
Senior user
Posts: 629
Joined: 2006-12-22 07:58
Location: Istanbul

Re: Account rule deletes message but not the reference

Post by katip » 2019-04-13 23:06

wait 10 ninutes or so. it doesn't come right after sending.
Katip
--
HMS 5.7.0-B2428-LTS-64-bit, MySQL 5.7.24, SA 3.4.2, ClamAV 0.101.2 + SaneS

User avatar
SorenR
Senior user
Senior user
Posts: 3063
Joined: 2006-08-21 15:38
Location: Denmark

Re: Account rule deletes message but not the reference

Post by SorenR » 2019-04-14 00:08

Hmm... You remember to check for "X-hMailServer-LoopCount Less than 1" in your forwarding rule?
SørenR.

The quantum rule of insecurity which states that the act of observing how vulnerable a host or service is changes the insecurity level of the service.

User avatar
katip
Senior user
Senior user
Posts: 629
Joined: 2006-12-22 07:58
Location: Istanbul

Re: Account rule deletes message but not the reference

Post by katip » 2019-04-14 05:51

SorenR wrote:
2019-04-14 00:08
Hmm... You remember to check for "X-hMailServer-LoopCount Less than 1" in your forwarding rule?
yes, makes no difference.
Katip
--
HMS 5.7.0-B2428-LTS-64-bit, MySQL 5.7.24, SA 3.4.2, ClamAV 0.101.2 + SaneS

Post Reply