Help: How to reduce "out of memory errors" - IMAP users
Help: How to reduce "out of memory errors" - IMAP users
I am consistently getting out of memory errors which is causing some glitches with imap users - emails not syncing for one. Errors start pouring in around 1.2-1.4 GB of ram usage
Is there anything I do with their folder structure to help with these errors?
Is having a inbox with a large number of emails cause large mem usage?
Will moving emails from inbox to various sub-folders help?
I have about 60 users total all using imap with 108 GB of files in the data folder.
Is there anything I do with their folder structure to help with these errors?
Is having a inbox with a large number of emails cause large mem usage?
Will moving emails from inbox to various sub-folders help?
I have about 60 users total all using imap with 108 GB of files in the data folder.
- jimimaseye
- Moderator
- Posts: 10060
- Joined: 2011-09-08 17:48
Re: Help: How to reduce "out of memory errors" - IMAP users
What version are you using? What PERFORMANCE settings do you have set (in ADVANCED).
This information will help people help you.
This information will help people help you.
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829
Re: Help: How to reduce "out of memory errors" - IMAP users
I have the most recent version - 5.6.4 build 2283. Windows 2012, dual xeon 2440, 16gb. I've been using hmailserver for 1 1/2 years with no issues until November 2015.
The performance settings do not affect my memory usage of any note - I've turned on indexing for 1 month, no change. Changed Max number of command threads (currently set to 50) ,Delivery threads(currently set to 50), Max number of asynchronous task threads(currently set to 20) -- again, no appreciable affect on memory usage. Caching is turned on with 30 seconds for all fields - I've tried a number of values here as well.
The performance settings do not affect my memory usage of any note - I've turned on indexing for 1 month, no change. Changed Max number of command threads (currently set to 50) ,Delivery threads(currently set to 50), Max number of asynchronous task threads(currently set to 20) -- again, no appreciable affect on memory usage. Caching is turned on with 30 seconds for all fields - I've tried a number of values here as well.
Re: Help: How to reduce "out of memory errors" - IMAP users
You don't need 50 command threads. The ideal number should be the number of physical cores or threads. Xeon 2440 comes with 6 cores, 12 threads. There are 12 cores, 24 threads in a dual Xeon 2440 server. Try 12 for command threads. Xeon 2440 is a fast processor, I would even reduce the number if there's no noticeable performance impact.
Re: Help: How to reduce "out of memory errors" - IMAP users
ALSO, what database are you using?
Where is your hMailserver DATA folder located?
Where is your hMailserver DATA folder located?
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation
https://www.hmailserver.com/documentation
Re: Help: How to reduce "out of memory errors" - IMAP users
MySQL ver. 5.5.34 running on the same physical server as hmailserver.
Re: Help: How to reduce "out of memory errors" - IMAP users
Seemed to be working well on 12 command threads for a half hour or so. Now back up to 1670 MB and getting out of memory errors again.
I did restart hmailserver when I changed to 12 cmd threads
I did restart hmailserver when I changed to 12 cmd threads
Re: Help: How to reduce "out of memory errors" - IMAP users
My hmailserver.exe uses only 130MB after 40 days of continuously running.
There are about 100 users in my hmailserver. The daily mail count is about 10,000. IMAP sessions during office hours is about 80 because some use POP3. The total size in Data folder is 20GB. MySql 64 bits, SpamAssassin and Clam AV services are on all the same machine.
Max number of command threads: 4
Delivery threads: 8
Max number of asynchronous task threads: 5
Single Xeon 2623 v3 (Quad cores 8 threads), 32GB RAM
Internet connection speed 15Mb down/2Mb up
---
Cache Enabled. All four values in the setting are 3600
Message indexing is NOT enabled.
Roundcube is also on the same machine.
There are about 100 users in my hmailserver. The daily mail count is about 10,000. IMAP sessions during office hours is about 80 because some use POP3. The total size in Data folder is 20GB. MySql 64 bits, SpamAssassin and Clam AV services are on all the same machine.
Max number of command threads: 4
Delivery threads: 8
Max number of asynchronous task threads: 5
Single Xeon 2623 v3 (Quad cores 8 threads), 32GB RAM
Internet connection speed 15Mb down/2Mb up
---
Cache Enabled. All four values in the setting are 3600
Message indexing is NOT enabled.
Roundcube is also on the same machine.
Last edited by tochi on 2016-01-11 21:58, edited 2 times in total.
- jimimaseye
- Moderator
- Posts: 10060
- Joined: 2011-09-08 17:48
Re: Help: How to reduce "out of memory errors" - IMAP users
So more IMAP users, plus POP3 users, and a lot lower performance settings (in some cases even lower than the default), with MySQL, SA and Clam all on the same server, all on a similar server and no problems.
Might be worth bcpaul just taking a punt and copying these values (and I would remove indexing).
Might be worth bcpaul just taking a punt and copying these values (and I would remove indexing).
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829
Re: Help: How to reduce "out of memory errors" - IMAP users
Just tried the settings above and ramps right back up to 1.7 - 1.8GB with memory errors.
I've only started getting these out of memory errors last Nov? after well over a year of having no issues.
No one's answered my question about IMAP inbox size - see my first post above.
Does a large number of inbox emails affect memory usage versus putting the emails into separate folders?? Some users have thousands of emails in their inbox. If I get inbox hell users to sort emails into folders, will this help?
Thanks
I've only started getting these out of memory errors last Nov? after well over a year of having no issues.
No one's answered my question about IMAP inbox size - see my first post above.
Does a large number of inbox emails affect memory usage versus putting the emails into separate folders?? Some users have thousands of emails in their inbox. If I get inbox hell users to sort emails into folders, will this help?
Thanks
Re: Help: How to reduce "out of memory errors" - IMAP users
if you have any users using webmail then make sure you have switched ON message indexing in your performance settings
- jimimaseye
- Moderator
- Posts: 10060
- Joined: 2011-09-08 17:48
Re: Help: How to reduce "out of memory errors" - IMAP users
I would imagine it matters on how many users are asking to look in to their Inbox at the same time. MY thinking is this: if someone is wanting an IMAP connection asking HMS to stream the headers of 10,000 emails in his inbox, then HMS is have to read the database of 10,000 emails, and link off to read the body content of them upon request (especially if they have 'offline storage' or indexing set in their client). Multiply that by 100 users, all with thousands of emails in their inbox. And if they are all logging on and doing it at the same time that would be a LOT of resource requirement for the HMS process, on disk activity and a LOT of reading of the database (also meaning a heavy load on the MySQL process). However, if those 100 users only have, say, 500 emails in their Inbox (because they operate a 'DEALT'/archive folder, for example) then when multiplied it is a LOT less intensive activity on the server. Stick on top of that the Indexing option then the HMS process spends its time constantly looking and updating the index with all of those 100 users doing their emails. (We know that indexing can significantly reduce performance if you dont have the correct hardware). I suspect, though, if those 10,000x100 emails were all in a subfolder (not the Inbox), and were only referred to occasionally then things would be a lot different.
I would be surprised if having 1,000,000 emails in 1 public Inbox is worse than having 100 users each wanting to read their own 10,000 emails (in fact I could imagine with caching and indexing it would be quicker ... but only just).
But its all relative of course. The numbers matter and all have to be taken in to consideration: Users, total emails in Inboxes, server processing power and resources, 'other' server activity at the same time......
But hey, what do I know?!
I would be surprised if having 1,000,000 emails in 1 public Inbox is worse than having 100 users each wanting to read their own 10,000 emails (in fact I could imagine with caching and indexing it would be quicker ... but only just).
But its all relative of course. The numbers matter and all have to be taken in to consideration: Users, total emails in Inboxes, server processing power and resources, 'other' server activity at the same time......
But hey, what do I know?!
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829
Re: Help: How to reduce "out of memory errors" - IMAP users
also, OPTIMIZE your database tables if you haven't already done so. You should stop hmailserver before doing this.
Re: Help: How to reduce "out of memory errors" - IMAP users
ALSO, just thinking aloud, what if one or more of the IMAP clients does offline sychronisation - a default with Thunderbird, then this would increase the overhead of the server immensely.
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation
https://www.hmailserver.com/documentation
Re: Help: How to reduce "out of memory errors" - IMAP users
Matt, could you elaborate on this?mattg wrote:ALSO, just thinking aloud, what if one or more of the IMAP clients does offline sychronisation - a default with Thunderbird, then this would increase the overhead of the server immensely.
I do have 2 clients on Thunderbird, 6 on Roundcube webmail, 7 on Apple Mail, ~ 50 on Outlook
- jimimaseye
- Moderator
- Posts: 10060
- Joined: 2011-09-08 17:48
Re: Help: How to reduce "out of memory errors" - IMAP users
What matt (and I, when I said above " especially if they have 'offline storage' or indexing set in their client.") means, is that these clients may well be downloading each and every email that is in the subscribed folders of the user down to the local email client/pc ('offline', so that they can still be read when there is no network connection). We know Outlook saves pretty much every bit of the email in to the local .OST file. Thunderbird also does the same (if you have "Keep Messages For This Account On My Computer" to in the account settings - 'synchronisation & Storage' section). Im sure apple mail will have a similar option (although I dont know about this client).
In any case it means that potentially you have 108GB of email data needing to be downloaded to these clients if they all have it set. And every time an email is moved to another folder, deleted, or written, then a copy of these emails has to be downloaded from the server to update these (subscribed) clients. You will also notice that in Outlook there is a 'Compact and Repair' option to redo this (usually huge) .OST file because the frequent changes can make them VERY inconsistent and poor in performance (see https://support.microsoft.com/en-us/kb/289987). Then this could also be calling upon the server to re-download messages whilst it re-jigs itself whilst its not in a decent compact state.
All of that download is database and disk I/O intensive....and who knows what memory implications. But I would bank on there being an effect.
In any case it means that potentially you have 108GB of email data needing to be downloaded to these clients if they all have it set. And every time an email is moved to another folder, deleted, or written, then a copy of these emails has to be downloaded from the server to update these (subscribed) clients. You will also notice that in Outlook there is a 'Compact and Repair' option to redo this (usually huge) .OST file because the frequent changes can make them VERY inconsistent and poor in performance (see https://support.microsoft.com/en-us/kb/289987). Then this could also be calling upon the server to re-download messages whilst it re-jigs itself whilst its not in a decent compact state.
All of that download is database and disk I/O intensive....and who knows what memory implications. But I would bank on there being an effect.
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829
Re: Help: How to reduce "out of memory errors" - IMAP users
The clients do all have offline storage, but I am assuming that only new/changed messages get downloaded to the client.
Matt said that this would increase the overhead of the server immensely.
So how/why would online sync be better? Seems backwards to me. Otherwise you would be using a glorified webmail app. In fact why would you even bother to use Outlook or Applemail -- just put all of your users on roundcube.
I am still having no success in reducing the out of memory errors. I will have to put hmailserver on the shelf soon if I cannot solve this -- sadly
Matt said that this would increase the overhead of the server immensely.
So how/why would online sync be better? Seems backwards to me. Otherwise you would be using a glorified webmail app. In fact why would you even bother to use Outlook or Applemail -- just put all of your users on roundcube.
I am still having no success in reducing the out of memory errors. I will have to put hmailserver on the shelf soon if I cannot solve this -- sadly
- jimimaseye
- Moderator
- Posts: 10060
- Joined: 2011-09-08 17:48
Re: Help: How to reduce "out of memory errors" - IMAP users
Well yes its true that only new emails would be downloaded BUT we dont know what the users are doing (have they all already completed initial downloads? Have you rebuilt the OST file? Is there an (outlook) client misbehaving?)bcpaul wrote:The clients do all have offline storage, but I am assuming that only new/changed messages get downloaded to the client.
Matt said that this would increase the overhead of the server immensely.
So how/why would online sync be better? Seems backwards to me. Otherwise you would be using a glorified webmail app. In fact why would you even bother to use Outlook or Applemail -- just put all of your users on roundcube.
Online or offline sync: there are benefits for both and Online sync cannot be discounted. For example, why would you want offline sync if you are running a desktop (non-mobile) email client that is permanently connected to the network/email server? By keeping online you have less overheads: less 'duplication' of data (same email messages stored in multiple places), network traffic, disk space/usage, huge OST files that you constantly need to rebuild! But of course its not the same as a webmail client because you still get the other benefits of your email client (such as built in calender functions etc). Of course, similarly, going OFFLINE means you can carry your laptop without being online and still see your emails and text/message searches are done my the client and not the server (less resource on the server) which may make things quicker.
So there are benefits for both - it depends on your needs.
Ive just realised, that you havent actually displayed the errors you are reporting (about out of memory). Where do you see them and can you post them here?
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829
Re: Help: How to reduce "out of memory errors" - IMAP users
The Disk ReadFile errors go hand-in-hand with the out of memory.
MailServer has run out of memory, clearing caches."
"APPLICATION" 12844 "2016-01-20 07:54:41.943" "OutOfMemoryHandler - hMailServer has run out of memory, clearing caches."
"APPLICATION" 12844 "2016-01-20 07:54:41.943" "OutOfMemoryHandler - hMailServer has run out of memory, clearing caches."
"ERROR" 12844 "2016-01-20 07:54:41.943" "Severity: 3 (Medium), Code: HM4227, Source: File::ReadFile, Description: An unknown error occurred while reading file from disk."
"ERROR" 12844 "2016-01-20 07:54:41.943" "Severity: 3 (Medium), Code: HM5042, Source: File::ReadTextFile, Description: An unknown error occurred while reading a text file from disk."
"ERROR" 12844 "2016-01-20 07:54:41.943" "Severity: 1 (Critical), Code: HM5062, Source: IMAPFetch::ReportCriticalError_, Description: ERROR HM10001 - IMAP FETCH: Error when loading MIME message. - C:\Program Files (x86)\hMailServer\Data\xxxx.com\username\B5\{B5E3A625-0D93-4389-92D8-16E5F271DBD4}.eml"
"APPLICATION" 12844 "2016-01-20 07:54:41.943" "OutOfMemoryHandler - hMailServer has run out of memory, clearing caches."
"APPLICATION" 12844 "2016-01-20 07:54:41.943" "OutOfMemoryHandler - hMailServer has run out of memory, clearing caches."
"ERROR" 12844 "2016-01-20 07:54:41.943" "Severity: 3 (Medium), Code: HM5136, Source: TCPConnection::AsyncReadCompleted, Description: An error occured while parsing data. Data length: 76, Data: 14.694 UID FETCH 5:33,35:63 (INTERNALDATE UID RFC822.SIZE FLAGS BODY.PEEK[]). Remote IP: 24.84.216.244"
"ERROR" 12844 "2016-01-20 07:54:41.943" "Severity: 2 (High), Code: HM4208, Source: ExceptionHandler::Run, Description: An error occured while executing 'IOCPQueueWorkerTask', Message: bad allocation"
"ERROR" 12844 "2016-01-20 07:54:42.896" "Severity: 1 (Critical), Code: HM5519, Source: StackLogger::Log, Description: An error has been detected. A mini dump has been written to C:\Program Files (x86)\hMailServer\Logs\minidump_2016-01-20 075441_{124122D8-2ADE-43C1-ABD9-A3405209A436}.dmp"
"SMTPD" 5316 5319 "2016-01-20 07:54:48.709" "24.84.216.244" "SENT: 220 Welcome to ark.net, go ahead."
MailServer has run out of memory, clearing caches."
"APPLICATION" 12844 "2016-01-20 07:54:41.943" "OutOfMemoryHandler - hMailServer has run out of memory, clearing caches."
"APPLICATION" 12844 "2016-01-20 07:54:41.943" "OutOfMemoryHandler - hMailServer has run out of memory, clearing caches."
"ERROR" 12844 "2016-01-20 07:54:41.943" "Severity: 3 (Medium), Code: HM4227, Source: File::ReadFile, Description: An unknown error occurred while reading file from disk."
"ERROR" 12844 "2016-01-20 07:54:41.943" "Severity: 3 (Medium), Code: HM5042, Source: File::ReadTextFile, Description: An unknown error occurred while reading a text file from disk."
"ERROR" 12844 "2016-01-20 07:54:41.943" "Severity: 1 (Critical), Code: HM5062, Source: IMAPFetch::ReportCriticalError_, Description: ERROR HM10001 - IMAP FETCH: Error when loading MIME message. - C:\Program Files (x86)\hMailServer\Data\xxxx.com\username\B5\{B5E3A625-0D93-4389-92D8-16E5F271DBD4}.eml"
"APPLICATION" 12844 "2016-01-20 07:54:41.943" "OutOfMemoryHandler - hMailServer has run out of memory, clearing caches."
"APPLICATION" 12844 "2016-01-20 07:54:41.943" "OutOfMemoryHandler - hMailServer has run out of memory, clearing caches."
"ERROR" 12844 "2016-01-20 07:54:41.943" "Severity: 3 (Medium), Code: HM5136, Source: TCPConnection::AsyncReadCompleted, Description: An error occured while parsing data. Data length: 76, Data: 14.694 UID FETCH 5:33,35:63 (INTERNALDATE UID RFC822.SIZE FLAGS BODY.PEEK[]). Remote IP: 24.84.216.244"
"ERROR" 12844 "2016-01-20 07:54:41.943" "Severity: 2 (High), Code: HM4208, Source: ExceptionHandler::Run, Description: An error occured while executing 'IOCPQueueWorkerTask', Message: bad allocation"
"ERROR" 12844 "2016-01-20 07:54:42.896" "Severity: 1 (Critical), Code: HM5519, Source: StackLogger::Log, Description: An error has been detected. A mini dump has been written to C:\Program Files (x86)\hMailServer\Logs\minidump_2016-01-20 075441_{124122D8-2ADE-43C1-ABD9-A3405209A436}.dmp"
"SMTPD" 5316 5319 "2016-01-20 07:54:48.709" "24.84.216.244" "SENT: 220 Welcome to ark.net, go ahead."
- jimimaseye
- Moderator
- Posts: 10060
- Joined: 2011-09-08 17:48
Re: Help: How to reduce "out of memory errors" - IMAP users
Hmmm... a quick google and it doesnt look like you are the first. I read somewhere that Martin "is looking in to it" (but cant find the post again). Might be good if he could comment here.
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829
Re: Help: How to reduce "out of memory errors" - IMAP users
I think I just resolved such case for my installation. I simply set problematic user with POP3 and downloaded all messages. Now the error is gone.
I know it's just a workaround, but it works for me. You can also leave user on IMAP, but move all his current messages to archive.
I have no idea why some message in that user's mailbox was causing error.
I know it's just a workaround, but it works for me. You can also leave user on IMAP, but move all his current messages to archive.
I have no idea why some message in that user's mailbox was causing error.
Re: Help: How to reduce "out of memory errors" - IMAP users
Yes it does, because the client software constantly checks ALL messages, not just when there are changes.bcpaul wrote:The clients do all have offline storage, but I am assuming that only new/changed messages get downloaded to the client.
Matt said that this would increase the overhead of the server immensely.
Changing to POP for this one account is the best solution by far.
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation
https://www.hmailserver.com/documentation
Re: Help: How to reduce "out of memory errors" - IMAP users
Okay, I've narrowed the problem down. Seems to be Apple Mail users that have purchased a new computer and have synced their email account from scratch. There are 3 people that have recently purchased new macbooks and have tried to re-download/offline sync all of their emails.
Syncing large number of emails at once seems to cause out-of-memory errors into perpetuity. They have large numbers of missing emails and syncing never seems to complete -- although apple mail says there are no new emails. Causes havoc with hmailserver.
The out-of-memory errors seem to originate from these apple mail users.
Not sure how to solve this?
Syncing large number of emails at once seems to cause out-of-memory errors into perpetuity. They have large numbers of missing emails and syncing never seems to complete -- although apple mail says there are no new emails. Causes havoc with hmailserver.
The out-of-memory errors seem to originate from these apple mail users.
Not sure how to solve this?
- jimimaseye
- Moderator
- Posts: 10060
- Joined: 2011-09-08 17:48
Re: Help: How to reduce "out of memory errors" - IMAP users
In december I made a point on each client (one at a time, though) to re-download (and reindex locally) all messages for local offline storage (Thunderbird - "repair folder"). Collectively there where about 3500 email messages in each folder with about 9 folders to do (up to 8.5 GB of email message data downloading). Dont know what these number seem to be like with your folders but I didnt have any IMAP problems. (And admitedly I was doing 1 client at a time - all clients have access to the multiple accounts thereby all doing the same job).
How does it compare?
Suggestion: as IT bod, get each macbook in the office and do the downloads one-by one?
How does it compare?
Suggestion: as IT bod, get each macbook in the office and do the downloads one-by one?
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829
Re: Help: How to reduce "out of memory errors" - IMAP users
SOLVED:
Avast Antivirus is the culprit on the Mac's. Got rid of it and voila no more syncing or out-of-memory issues.
How come I can't edit my old posts??? Would like to change the topic name to SOLVED How to reduce "out of memory errors" - IMAP users
Avast Antivirus is the culprit on the Mac's. Got rid of it and voila no more syncing or out-of-memory issues.
How come I can't edit my old posts??? Would like to change the topic name to SOLVED How to reduce "out of memory errors" - IMAP users
Re: Help: How to reduce "out of memory errors" - IMAP users
Arg...can't edit again.
Avast seems to work day-to-day, but if a user was re-syncing everything, then Avast causes extreme load (quickly causes out of memory errors) on hmailserver and WILL not ever sync correctly -- just takes one user to cause havoc
Avast seems to work day-to-day, but if a user was re-syncing everything, then Avast causes extreme load (quickly causes out of memory errors) on hmailserver and WILL not ever sync correctly -- just takes one user to cause havoc
Re: Help: How to reduce "out of memory errors" - IMAP users
Thanks for posting your answer
Editing of own posts is time defined (ie only next 30 minutes I believe)
We don't normally mark threads as solved, or lock them
Glad that you got this sorted
( i dare say that this would be for all Anti-virus that inspect mail in real time)
Editing of own posts is time defined (ie only next 30 minutes I believe)
We don't normally mark threads as solved, or lock them
Glad that you got this sorted
( i dare say that this would be for all Anti-virus that inspect mail in real time)
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation
https://www.hmailserver.com/documentation
Re: Help: How to reduce "out of memory errors" - IMAP users
I hope the developers of hmailserver can solve the issue from server end. A robust program should be able to defend itself from requests made by ill-behaved clients.
It will be very helpful if you can give the procedures to reproduce this issue and list detailed info like versions of all software involved.
It will be very helpful if you can give the procedures to reproduce this issue and list detailed info like versions of all software involved.
Re: Help: How to reduce "out of memory errors" - IMAP users
Client: Tested on Mac OS 10 El Capitan (all updates as of Jan 27, 2016). Avast Free Antivirus.
Server: hmailserver 5.6.4 B-2284 running on Windows 2012, HP Proliant ML 350 Gen 8 (from 2014)
If a Mac user (on iMac or Macbook) re-creates their email account (new computer, re-format, or refresh account). Seen on Macbook 2012, 2015, iMac 2013.
Ports IMAP 997 with SSL/TLS, SMTP 587 (STARTTLS optional)
Affected clients have approx 10-20,000 emails in their account (~5GB). From a couple to about 60 IMAP folders.
Hmailserver performance settings have NO effect on this out-of-memory issue
It just takes one user to wreak havok on ALL of your email users.
Out-of-memory errors cause the following on all clients:
1. Emails do not come in live (IMAP). Have to close Outlook, Applemail and re-open, then emails are received.
2. You may get emails coming in with NO SUBJECT, No Content (body), No Sender.
3. The Mac users with Avast get stuck on syncing, never completing. Even one user will quickly hit the memory limit of hmailserver.
4. You will need to constantly restart the hmailserver service to attempt to have a functional email server. This works for a short period of time.
SOLUTION: Un-install AVAST anti-virus on the offending Macs. I deleted the email account, then re-created it to resync it. You may not need to do this?
Not sure if this issue also affects PC with Avast, or other anti-virus software
Hopefully, Matt can add something in hmailserver to detect this stuck sync process.
Server: hmailserver 5.6.4 B-2284 running on Windows 2012, HP Proliant ML 350 Gen 8 (from 2014)
If a Mac user (on iMac or Macbook) re-creates their email account (new computer, re-format, or refresh account). Seen on Macbook 2012, 2015, iMac 2013.
Ports IMAP 997 with SSL/TLS, SMTP 587 (STARTTLS optional)
Affected clients have approx 10-20,000 emails in their account (~5GB). From a couple to about 60 IMAP folders.
Hmailserver performance settings have NO effect on this out-of-memory issue
It just takes one user to wreak havok on ALL of your email users.
Out-of-memory errors cause the following on all clients:
1. Emails do not come in live (IMAP). Have to close Outlook, Applemail and re-open, then emails are received.
2. You may get emails coming in with NO SUBJECT, No Content (body), No Sender.
3. The Mac users with Avast get stuck on syncing, never completing. Even one user will quickly hit the memory limit of hmailserver.
4. You will need to constantly restart the hmailserver service to attempt to have a functional email server. This works for a short period of time.
SOLUTION: Un-install AVAST anti-virus on the offending Macs. I deleted the email account, then re-created it to resync it. You may not need to do this?
Not sure if this issue also affects PC with Avast, or other anti-virus software
Hopefully, Matt can add something in hmailserver to detect this stuck sync process.
Re: Help: How to reduce "out of memory errors" - IMAP users
Would it be possible for you to send the following things to me:
- The hMailServer log for 2016-01-20
- The dump file (C:\Program Files (x86)\hMailServer\Logs\minidump_2016-01-20 075441_{124122D8-2ADE-43C1-ABD9-A3405209A436}.dmp)
- The message file (C:\Program Files (x86)\hMailServer\Data\xxxx.com\username\B5\{B5E3A625-0D93-4389-92D8-16E5F271DBD4}.eml)
This way I can hopefully see what's causing the crash.
My address is martin@hmailserver.com
- The hMailServer log for 2016-01-20
- The dump file (C:\Program Files (x86)\hMailServer\Logs\minidump_2016-01-20 075441_{124122D8-2ADE-43C1-ABD9-A3405209A436}.dmp)
- The message file (C:\Program Files (x86)\hMailServer\Data\xxxx.com\username\B5\{B5E3A625-0D93-4389-92D8-16E5F271DBD4}.eml)
This way I can hopefully see what's causing the crash.
My address is martin@hmailserver.com
Martin Knafve
Re: Help: How to reduce "out of memory errors" - IMAP users
After the change, how much memory usage are you seeing?
Would it be possible for you to send the log so that I can see what command the client issued caused the problem?
Would it be possible for you to send the log so that I can see what command the client issued caused the problem?
Martin Knafve
Re: Help: How to reduce "out of memory errors" - IMAP users
Mi Martin,
Do you have an alternate email address? Tried emailing, but bouncing back.
Typical mem usage now is ~130-200 MB. Great
I will be testing an alternative - AVG? Will get back.
Do you have an alternate email address? Tried emailing, but bouncing back.
Typical mem usage now is ~130-200 MB. Great
I will be testing an alternative - AVG? Will get back.
Re: Help: How to reduce "out of memory errors" - IMAP users
I've done a bit more testing:
AVAST does cause issues with some Mac users - emails missing. But no out of memory errors.
I have tested Avira and AVG anti-virus and I did not notice any issues. So ALL of our mac users are required to uninstall AVAST and use Avira or AVG as an alternate.
NO out of memory or other issues since.
AVAST does cause issues with some Mac users - emails missing. But no out of memory errors.
I have tested Avira and AVG anti-virus and I did not notice any issues. So ALL of our mac users are required to uninstall AVAST and use Avira or AVG as an alternate.
NO out of memory or other issues since.