iOS devices do not purge deleted mails

Use this forum if you have installed hMailServer and want to ask a question related to a production release of hMailServer. Before posting, please read the troubleshooting guide. A large part of all reported issues are already described in detail here.
Post Reply
0x008
New user
New user
Posts: 4
Joined: 2018-11-25 14:00

iOS devices do not purge deleted mails

Post by 0x008 » 2018-11-25 14:28

Hi all,

I have many clients with iOS devices. Unfortunately many (or all) of them do not purge deleted mails and they send mails like "My Inbox is full but I deleted all mails in inbox and bin". My short-term solution untill now was to increase the inbox size of the account, but there must be a better way to fix it.

I tried to recreate the problem as folllows:

1. Added a brand new Account to one Domain.
2. Added the account on my new iPad. I set the "Deleted messages -> Remove" Setting to "One Day".
3. Sent a mail from another account to this account. Mailbox size: 0.005 MB
4. Deleted the mail in the inbox. Mailbox size: 0.01 MB
5. ImapSize shows me two deleted (crossed out) mails in INBOX and Deleted Messages.
6. The size of the account is 0.01 MB even after a week.

When doing the same procedure in Outlook, everything works and my mailbox is right back at 0 MB.

Could someone give me an advice how to solve this problem?

Thanks in advance!

User avatar
mattg
Moderator
Moderator
Posts: 20788
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: iOS devices do not purge deleted mails

Post by mattg » 2018-11-26 02:40

A 'purge' or 'expunge' is required to be sent from the apple device, but this apparently is tricky to achieve with iOS devices - lots on the internet about it.

You could possibly script a method to expunge folders manually overnight, but that may have consequences as many users opt to leave their trash as a last search resort.

Perhaps you could do a script just for specific accounts.

I find that roundcube handles expunges really well, and even just logging into the account with roundcube and swapping folders inbox>> trash >> inbox is enough to force a purge/expunge of both folders
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

Virinum
Normal user
Normal user
Posts: 69
Joined: 2018-11-23 14:42
Location: Germany

Re: iOS devices do not purge deleted mails

Post by Virinum » 2018-11-26 11:15

Yes, iOS doesn't like to purge.
I noticed that a mail is copied to trash when you delete it on an iOS-Device (like the most clients do). So you have this e-mail twice. One copy e.g. in INBOX (but with flag "deleted") and one copy in trash. I've wrote a script which runs weekly and deletes all messages that are flagged "deleted".

Create a .vbs file and call it with the windows task scheduler:

Code: Select all

Const HMSADMINUSER = "Administrator"
Const HMSADMINPWD = "PASSWORD"

'Objects
Dim oApp, oDomain, oAccount, oFolder, oMessages

'Array
Dim DeleteMessages

'Numeric
Dim MessageID


Set oApp = CreateObject("hMailServer.Application")
Call oApp.Authenticate(HMSADMINUSER, HMSADMINPWD)

Set DeleteMessages = CreateObject("System.Collections.ArrayList")

For w = 0 to oApp.Domains.Count - 1
  Set oDomain = oApp.Domains.Item(w)  
  For x = 0 to oDomain.Accounts.Count - 1
    Set oAccount = oDomain.Accounts.Item(x)
    CheckFolders(oAccount.IMAPFolders)
  Next
Next

Function CheckFolders(oFolders)
  For y = 0 to oFolders.Count - 1
    Set oFolder = oFolders.Item(y)
    Set oMessages  = oFolder.Messages
    DeleteMessages.Clear
    For z = 0 to oMessages.Count - 1
      MessageID = oMessages.Item(z).ID
      If (oMessages.ItemByDBID(MessageID).Flag(2)) Then
        oMessages.ItemByDBID(MessageID).Flag(2) = False
        DeleteMessages.Add MessageID
      End If
    Next
    For Each element In DeleteMessages
      'WScript.Echo oMessages.ItemByDBID(element).Subject
      oMessages.DeleteByDBID(element)
    Next
    CheckFolders(oFolder.SubFolders)
  Next
End Function

User avatar
SorenR
Senior user
Senior user
Posts: 3562
Joined: 2006-08-21 15:38
Location: Denmark

Re: iOS devices do not purge deleted mails

Post by SorenR » 2018-11-26 23:35

Virinum wrote:
2018-11-26 11:15
Yes, iOS doesn't like to purge.
I noticed that a mail is copied to trash when you delete it on an iOS-Device (like the most clients do). So you have this e-mail twice. One copy e.g. in INBOX (but with flag "deleted") and one copy in trash. I've wrote a script which runs weekly and deletes all messages that are flagged "deleted".
Nice 8)

I see you used Martin's recursive folder list script as basis. [thumbs up]
SørenR.

“With age comes wisdom, but sometimes age comes alone.”
- Oscar Wilde

0x008
New user
New user
Posts: 4
Joined: 2018-11-25 14:00

Re: iOS devices do not purge deleted mails

Post by 0x008 » 2018-12-02 22:15

Thanks for the script!

I tested it in my staging environment, logging the ids of the messages to be deleted for a testaccount.

Code: Select all

select * from hm_messages where messageaccountid = 123
returns one mail with messageflag=67.
According to the eMessageFlag Enum this should be virus scan (64) + deleted (2) + seen (1).

Unfortunately no messageid gets logged / deleted by the script.
It seems that

Code: Select all

oMessages.ItemByDBID(MessageID).Flag(2)
resolves for 67 to false.

But why? Running the script through all accounts didn´t result in a logged / deleted message id either:

Code: Select all

select * from hm_messages where messageflags = 3
Returns 94 mails which are seen (1) and deleted (2). The script loggs none of them...

Could someone give me a hint?

Thanks!

User avatar
Dravion
Senior user
Senior user
Posts: 1688
Joined: 2015-09-26 11:50
Location: Germany
Contact:

Re: iOS devices do not purge deleted mails

Post by Dravion » 2018-12-02 22:51

IMAP RFC compliant would be to simply login via telnet / scrpt and issue the EXPUNGE Command
periodically. Problem, yoi need the Inbox Credentials in you Script for eveey User.

IMHO the vest way to do it, is using the COM API
and deleting Messages with the correct State ID.

Virinum
Normal user
Normal user
Posts: 69
Joined: 2018-11-23 14:42
Location: Germany

Re: iOS devices do not purge deleted mails

Post by Virinum » 2018-12-02 23:12

0x008 wrote:
2018-12-02 22:15
I tested it in my staging environment, logging the ids of the messages to be deleted for a testaccount.
I can see why you are looking into the database. But I would just use the COM API. Delete a message on an iOS-device and run the script below. You should see the subject of the message you just deleted. The script I provided before is working pretty fine on my server.

Here the script for testing:

Code: Select all

Const HMSADMINUSER = "Administrator"
Const HMSADMINPWD = "PASSWORD"

'Objects
Dim oApp, oDomain, oAccount, oFolder, oMessages

'Array
Dim DeleteMessages

'Numeric
Dim MessageID


Set oApp = CreateObject("hMailServer.Application")
Call oApp.Authenticate(HMSADMINUSER, HMSADMINPWD)

Set DeleteMessages = CreateObject("System.Collections.ArrayList")

For w = 0 to oApp.Domains.Count - 1
  Set oDomain = oApp.Domains.Item(w)  
  For x = 0 to oDomain.Accounts.Count - 1
    Set oAccount = oDomain.Accounts.Item(x)
    CheckFolders(oAccount.IMAPFolders)
  Next
Next

Function CheckFolders(oFolders)
  For y = 0 to oFolders.Count - 1
    Set oFolder = oFolders.Item(y)
    Set oMessages  = oFolder.Messages
    DeleteMessages.Clear
    For z = 0 to oMessages.Count - 1
      MessageID = oMessages.Item(z).ID
      If (oMessages.ItemByDBID(MessageID).Flag(2)) Then
        DeleteMessages.Add MessageID
      End If
    Next
    For Each element In DeleteMessages
      WScript.Echo oMessages.ItemByDBID(element).Subject
    Next
    CheckFolders(oFolder.SubFolders)
  Next
End Function

User avatar
Dravion
Senior user
Senior user
Posts: 1688
Joined: 2015-09-26 11:50
Location: Germany
Contact:

Re: iOS devices do not purge deleted mails

Post by Dravion » 2018-12-03 12:23

The hMailServer internal states of a Message (POP3 and IMAP neutral)

Code: Select all

[uuid(90745436-4C3F-11D9-AD17-A0BCEA20CD09),
	helpstring("Message flags")] 
	enum 
{
	[helpstring("Seen flag")] 
	eMFSeen = 1,
	[helpstring("Deleted flag")] 
	eMFDeleted = 2,
	[helpstring("Flagged flag")] 
	eMFFlagged = 4,
	[helpstring("Answered flag")] 
	eMFAnswered = 8,
	[helpstring("Draft flag")] 
	eMFDraft = 16,
	[helpstring("Recent flag")] 
	eMFRecent = 32,
	[helpstring("Virus scan")] 
	eMFVirusScan = 64,
} eMessageFlag;

0x008
New user
New user
Posts: 4
Joined: 2018-11-25 14:00

Re: iOS devices do not purge deleted mails

Post by 0x008 » 2018-12-03 20:56

I cleared the account, sent a new mail and deleted it on my ipad in inbox and bin. Afterwards I checked the database with

Code: Select all

select * from hm_messages where messageaccountid = 123
and two rows returned. One with messageflags=67, one with messageflags=99. The data folder of my testing account contains two .eml files as well. Running Virinum´s testing script returns no result.

Referring to the eMessageFlags:
67 = eMFVirusScan + eMFDeleted + eMFSeen
and
99 = eMFVirusScan + eMFRecent + eMFDeleted + eMFSeen

Both messages should resolve oMessages.ItemByDBID(MessageID).Flag(2) to true and the subject should be logged, but no subject is logged.

I don´t get it.

User avatar
Dravion
Senior user
Senior user
Posts: 1688
Joined: 2015-09-26 11:50
Location: Germany
Contact:

Re: iOS devices do not purge deleted mails

Post by Dravion » 2018-12-03 21:26

Could be a Mismarch between IMAP and hMailServers internal native States. In his script he references to IMAP Message flags, but you try to check the state against the COM API Native Message flags which is the underlying foundation of POP3 and IMAP Message flags.

It also could be a effect of deffered (asnyc I/O)
or a cached value which arrent saved to the DB at the tine you externaly queryung the tabke vua SQL.

1) Execute your script again
2) Stop/Start hMailServee service
3) Run your SQL-Query again

This should make sure that no deferred I/O or
chached records sets caused the confusion.

If its still the same, change the script to compare to COM API internal Data and Function but dont intermix IMAP and internal.

0x008
New user
New user
Posts: 4
Joined: 2018-11-25 14:00

Re: iOS devices do not purge deleted mails

Post by 0x008 » 2018-12-03 21:47

Unbelievable, restarting the hMailServer service was the solution!

Thanks to all of you for helping!

Virinum
Normal user
Normal user
Posts: 69
Joined: 2018-11-23 14:42
Location: Germany

Re: iOS devices do not purge deleted mails

Post by Virinum » 2018-12-03 21:59

0x008 wrote:
2018-12-03 21:47
Unbelievable, restarting the hMailServer service was the solution!

Thanks to all of you for helping!
I'm glad it works now.

But for the protocol here is what I've tested:

1. Send two test messages
2. Delete them from INBOX with an iPhone
3. Run the script

Result:
screenshot.png
screenshot.png (6.46 KiB) Viewed 2318 times
No restarting of the service needed.

User avatar
jimimaseye
Moderator
Moderator
Posts: 8518
Joined: 2011-09-08 17:48

Re: iOS devices do not purge deleted mails

Post by jimimaseye » 2018-12-03 23:53

The messages are subject to the cache of both the database and the folder of the email client. If you close of or drop the email client connection then only one of the email files would appear. Similarly restarting the service (and therefore the cache) also releases the files.

[Entered by mobile. Excuse my spelling.]
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

Post Reply