Cleaning up huge inbox

Use this forum if you have installed hMailServer and want to ask a question related to a production release of hMailServer. Before posting, please read the troubleshooting guide. A large part of all reported issues are already described in detail here.
Post Reply
lacazet2k
New user
New user
Posts: 1
Joined: 2020-02-19 09:16

Cleaning up huge inbox

Post by lacazet2k » 2020-02-19 09:18

I have a user with at least 200K messages in her inbox. Yes, 200 thousand. She is OK with moving all of these out into different folders, but the sheer volume poses a problem. Thunderbird and Outlook can't reliably handle this many emails. Frankly, Outlook 2010 chokes after a couple thousand. Is there a lightweight client, or a script, I can use to move all these emails? I would just use the Hmailserver admin rename to at least make the inbox usable, but the inbox is the one folder than can't be renamed.
tks

User avatar
Dravion
Senior user
Senior user
Posts: 1688
Joined: 2015-09-26 11:50
Location: Germany
Contact:

Re: Cleaning up huge inbox

Post by Dravion » 2020-02-19 10:34

You can reduce the Volume by moving for example
old Emails to another Folder and having only a few in your Inbox.

You also can delete old Emails completely in Thunderbird or via a Script on hMailServer side.

You can also increase the Hardware performance on the Server and Client Computer by using fast SSDs instead of old Harddrives.

palinka
Senior user
Senior user
Posts: 1936
Joined: 2017-09-12 17:57

Re: Cleaning up huge inbox

Post by palinka » 2020-02-19 15:02

lacazet2k wrote:
2020-02-19 09:18
I have a user with at least 200K messages in her inbox. Yes, 200 thousand. She is OK with moving all of these out into different folders, but the sheer volume poses a problem. Thunderbird and Outlook can't reliably handle this many emails. Frankly, Outlook 2010 chokes after a couple thousand. Is there a lightweight client, or a script, I can use to move all these emails? I would just use the Hmailserver admin rename to at least make the inbox usable, but the inbox is the one folder than can't be renamed.
tks
I had to move a very large mailbox a couple of times. Thunderbird choked the first time. Then, several months later, Thunderbird worked flawlessly. The difference? The first try was using Thunderbird portable. The second was the regular windows installer version. I don't know why, but it made a huge difference.

User avatar
jimimaseye
Moderator
Moderator
Posts: 8551
Joined: 2011-09-08 17:48

Re: Cleaning up huge inbox

Post by jimimaseye » 2020-02-19 15:49

palinka wrote:
2020-02-19 15:02
I had to move a very large mailbox a couple of times. Thunderbird choked the first time. Then, several months later, Thunderbird worked flawlessly. The difference? The first try was using Thunderbird portable. The second was the regular windows installer version. I don't know why, but it made a huge difference.
Very much coincidence but not related. Other factors may well be at play such as server and network activity. Email movements and deletions are simple imap commands sent by the client with the work being done by the server . Thunderbird (both versions) has the same performance and the speed 'choke' is the server movement or deleting of the the email files and updating the database accordingly... for each one as it processes them. Then they're is the feedback to thunderbird which the update which also relies on the network.

[Entered by mobile. Excuse my spelling.]
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

palinka
Senior user
Senior user
Posts: 1936
Joined: 2017-09-12 17:57

Re: Cleaning up huge inbox

Post by palinka » 2020-02-19 16:03

jimimaseye wrote:
2020-02-19 15:49
palinka wrote:
2020-02-19 15:02
I had to move a very large mailbox a couple of times. Thunderbird choked the first time. Then, several months later, Thunderbird worked flawlessly. The difference? The first try was using Thunderbird portable. The second was the regular windows installer version. I don't know why, but it made a huge difference.
Very much coincidence but not related. Other factors may well be at play such as server and network activity. Email movements and deletions are simple imap commands sent by the client with the work being done by the server . Thunderbird (both versions) has the same performance and the speed 'choke' is the server movement or deleting of the the email files and updating the database accordingly... for each one as it processes them. Then they're is the feedback to thunderbird which the update which also relies on the network.

[Entered by mobile. Excuse my spelling.]
What I meant by "choke" was that it would simply stop after a couple hundred messages as if everything were normal. Nothing to do with speed. I finally had to finish the operation in small batches. Very time consuming and frustrating. This was happening both at the source server (gmail) and the destination server (hmailserver).

The other time it was the same 2 servers and the operation was flawless with tens of thousands of messages getting moved uninterrupted. The only difference being portable vs installed versions. Coincidence? Possibly, but I doubt it. If the first try worked with gmail but not hmailserver or vice verse, then I would say the possibility of other factors is much greater. Reliability appears to be much greater with the installed version.

User avatar
mattg
Moderator
Moderator
Posts: 20799
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Cleaning up huge inbox

Post by mattg » 2020-02-22 03:18

I've had a play and written a script

It sorts messages older than 'daysago' into folders for the year and month in their date stamp. All as sub folders of the inbox.

eg

Inbox > 2017 > 01
Inbox > 2017 > 02
Inbox > 2017 > 03
Inbox > 2017 > 04


You will need to subscribe to the subfolders once only in the mail client

If the date in the message header isn't consistent, the message is skipped

tested on a couple of mail accounts with about than 10 000 messages, takes about an hour on my system.

Set the const at the top as required

Code: Select all

Option Explicit

Const HMSADMINUSER = "administrator"
Const HMSADMINPWD = "TopSecretAdminPassword"
Const OAccount = "test@example.com"
Const DaysAgo = 200

Dim oApp, oDomain, oFolder, oDate, i, j, oFolders, oMonth

Set oApp = CreateObject("hMailServer.Application")
Call oApp.Authenticate(HMSADMINUSER, HMSADMINPWD)

oDomain = Right(OAccount, Len(OAccount) - InStr(OAccount, "@"))
set oFolders = oApp.Domains.ItemByName(oDomain).Accounts.ItemByAddress(oAccount).IMAPFolders.itembyname("Inbox")

For i = oFolders.messages.count-1 to 0 Step -1
	On Error Resume Next
		oDate = CDate(RTrim(Mid(oFolders.messages.Item(i).date,6,11)))
		If now()- oDate > DaysAgo	Then 
			oFolder = "a"
			If oFolders.SubFolders.Count > 0 Then
				For j = 0 To oFolders.SubFolders.Count -1
					If oFolders.SubFolders.Item(j).Name = cStr(Year(oDate)) Then 
						oFolder = oFolders.SubFolders.itembyName(Year(oDate))
					End If
				Next 'j	
			End If	
			If oFolder="a" Then 
				oFolder = oFolders.subfolders.Add(Year(oDate))
			End If
			oFolder = "a"
			oMonth = Month(oDate)
			If oMonth <10 Then oMonth = "0" & CStr(Month(oDate))
			If oFolders.SubFolders.ItemByName(Year(oDate)).SubFolders.Count > 0 Then
				For j = 0 To oFolders.SubFolders.ItemByName(Year(oDate)).subfolders.Count -1
					If oFolders.SubFolders.ItemByName(Year(oDate)).subfolders.Item(j).Name = CStr(oMonth) Then 
						oFolder = oFolders.SubFolders.ItemByName(Year(oDate)).subfolders.itembyname(oMonth)
					End If
				Next 'j
			End if	
			If oFolder="a" Then 
				oFolder = oFolders.subfolders.ItemByName(Year(oDate)).subfolders.Add(oMonth)
			End If
			Call oFolders.messages.Item(i).copy(oFolder)
			call oFolders.Messages.DeleteByDBID(oFolders.messages.Item(i).ID)	
		End if
	On Error GoTo 0
Next ' message
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

Post Reply