Copying large, over 50MB, messages to new hMailServer mailbox via IMAP

Use this forum if you have installed hMailServer and want to ask a question related to a production release of hMailServer. Before posting, please read the troubleshooting guide. A large part of all reported issues are already described in detail here.
Post Reply
justinh
New user
New user
Posts: 3
Joined: 2019-08-15 19:57

Copying large, over 50MB, messages to new hMailServer mailbox via IMAP

Post by justinh » 2019-08-15 20:20

Hello Group,

I am helping a customer migrate from an old GroupWise email server to hMailServer. We discovered the best and cheapest route to tackle this is to use IMAP and connect to both servers. We are using Thunderbird as the client because we had many IMAP issues with Outlook previously. We are copying folders from the old GroupWise account to the new hMailServer account. Everything is going well, except extremely large messages are not being copied over. We increased the Thunderbird local cache to the max at 1GB and were able to transfer 51.7MB messages. We were not able to transfer anything larger at yet.

The hMailServer has mostly default settings and we started off leaving all limits to 0 (zero). We thought that if we for some reason change the message size to something ridiculous, that may help. So under the domain, we set the message size to 4GB or 4096000KB. So far, none of that helped. I turned IMAP logging on for Thunderbird which is a complicated log with thousands of lines of logs in a matter of seconds, but it looks like the message is sent to the hMailServer. I ran Wireshark and again, it "looks" like the message is sent. On the hMailServer, I see a portion in the log were it states that a message was sent to the specific folder, but it not there.

"IMAPD" 4000 149 "2019-08-15 13:29:46.543" "192.168.0.106" "SENT: * OK IMAPrev1"
"IMAPD" 3980 149 "2019-08-15 13:29:46.559" "192.168.0.106" "RECEIVED: 1 capability"
"IMAPD" 3980 149 "2019-08-15 13:29:46.559" "192.168.0.106" "SENT: * CAPABILITY IMAP4 IMAP4rev1 CHILDREN IDLE QUOTA SORT ACL NAMESPACE RIGHTS=texk[nl]1 OK CAPABILITY completed"
"IMAPD" 4000 149 "2019-08-15 13:29:46.575" "192.168.0.106" "RECEIVED: 3 login "user@domain.com" ***"
"IMAPD" 4000 149 "2019-08-15 13:29:46.606" "192.168.0.106" "SENT: 3 OK LOGIN completed"
"IMAPD" 3980 149 "2019-08-15 13:29:46.606" "192.168.0.106" "RECEIVED: 4 append "Cabinet/Good Files from Trash Folder" "24-Dec-2018 13:47:59 -0400" {122378810}"
"IMAPD" 3980 149 "2019-08-15 13:29:46.621" "192.168.0.106" "SENT: + Ready for literal data"

The message was not moved though. I tried to use the local folder in Thunderbird, which I can move very large messages to the local folder, but not to the hMailServer IMAP folders. I put Thunderbird offline, move the message successfully to the hMailServer IMAP folder, but once put back online, the message disappears from the IMAP folder. My customer has some extremely large email messages...over 2GB. He saves every email that is related to his business and we need to find a way that allows us to transfer there messages.

Thanks in advance, Justin

User avatar
jimimaseye
Moderator
Moderator
Posts: 8094
Joined: 2011-09-08 17:48

Re: Copying large, over 50MB, messages to new hMailServer mailbox via IMAP

Post by jimimaseye » 2019-08-15 20:47

HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

justinh
New user
New user
Posts: 3
Joined: 2019-08-15 19:57

Re: Copying large, over 50MB, messages to new hMailServer mailbox via IMAP

Post by justinh » 2019-08-15 20:53

jimimaseye,

I read over that before we even tried to start this project. The issue is that any message larger than 50MB is not moving over using IMAP. I was able to get on message transferred several times at 51.7MB, but nothing larger.

Debug code for my last attempt was:

"DEBUG" 4008 "2019-08-15 14:46:07.929" "The read operation failed. Bytes transferred: 0 Remote IP: 192.168.0.106, Session: 202, Code: 2, Message: End of file"
"DEBUG" 4008 "2019-08-15 14:46:07.944" "Ending session 202"

Any idea regarding the failed read operation?

Justin

User avatar
jimimaseye
Moderator
Moderator
Posts: 8094
Joined: 2011-09-08 17:48

Re: Copying large, over 50MB, messages to new hMailServer mailbox via IMAP

Post by jimimaseye » 2019-08-15 21:57

You tried imapcopy?
HMS 5.6.6 B2383 on Win Server 2008 R2 Foundation, + 5.6.7-B2415 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
mattg
Moderator
Moderator
Posts: 20024
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Copying large, over 50MB, messages to new hMailServer mailbox via IMAP

Post by mattg » 2019-08-16 00:29

There is Thunderbird 'chunk sizes' and IMAP timeouts tyo play with under the hood of Thunderbird.
Have you tried another mail client?

Outlook is crap at lots of things, but is often good at moving large messages. Set timeout to maximum
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

justinh
New user
New user
Posts: 3
Joined: 2019-08-15 19:57

Re: Copying large, over 50MB, messages to new hMailServer mailbox via IMAP

Post by justinh » 2019-08-16 17:43

jimimaseye, I never heard of imapcopy, but will look into that.

mattg, aside from Outlook and TBird, we have not tried any other client. The "look and feel" of TBird best suites my customer's needs, but we shall look into that. If not to just get the emails transferred over.

I thank the two of you and appreciate your time. I will report back in a few days with an update.

Justin

Post Reply