IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Use this forum if you want to discuss a problem or ask a question related to a hMailServer beta release.
Post Reply
estas
New user
New user
Posts: 7
Joined: 2015-09-02 11:05
Location: Belgium

IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by estas » 2015-09-02 16:06

Looking in other posts on this forum, this issue regarding Outlook and the UID error is mentioned a couple of times without any solution. The best way someone explained the problem was at topic 13361 (viewtopic.php?t=13361)
We have tried and searched for a solution but we are running out of time and have to find another way to give our employees access to our mails with IMAP. Somewhere in the topic I’m referring to, someone mentioned following
Bill48105 wrote:chrisso think you missed the point. Perhaps what you tell us is a CLUE in the underlying cause but the problem existed before Outlook 2010.. In fact with 2007 as others reported. So the problem is not 2010. It seems what you point out gives hints & confirms what we've suspected for a while now so it could still be useful in tracking down the root cause. The UID's get out of order or are different in terms of what Outlook thinks they should be. For example during the move to trash outlook might assume the UID remains the same but hmail changes it during the move. Or vice versa. We've never been able to reproduce it consistently enough to track what is happening so maybe what you post will help with that. If we can reproduce the problem we can try to fix it so thanks for the info.

Btw it is entirely possible the issue is a race condition. If you get time you might try one of the latest experimental builds (link in my signature) even if on a test bed. During development of STARTTLS we found a few spots in code where there were race conditions that resulted in corrupt emails and crashes. We have confirmation from a few people that the changes have solved crashes that happen in B9150 etc so you might give it a try. It's possible the threads get out of sync resulting in UID's being out of order.
Bill
If we would read the instructions regarding IMAP standards (http://tools.ietf.org/html/rfc3501) than following is very important: On section 2.3.1.1 it says:
The unique identifier of a message MUST NOT change during the session, and SHOULD NOT change between sessions.
We find it hard to believe that no one can simulate the problem while we can do it every time, again and again. It’s just not workable and if this issue cannot be solved, we will have to look for another mailserver.

The question is: is someone working on this problem?

User avatar
mattg
Moderator
Moderator
Posts: 22435
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by mattg » 2015-09-02 16:20

estas wrote:We find it hard to believe that no one can simulate the problem while we can do it every time, again and again.
Great

Then tell me what you do, so that we can reproduce it on our servers...


This is ONLY an issue with Outlook using IMAP, and is ONLY an issue at random times, not consistently every time that outlook is used.
FWIW Outlook 2013 has bigger IMAP issues, like only showing some of the available mail.
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

estas
New user
New user
Posts: 7
Joined: 2015-09-02 11:05
Location: Belgium

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by estas » 2015-09-02 16:59

That is an easy answer. Use hmailserver with 3 clients (outlook 2010). Connect them to the same account. Do things like: delete mails, move mails empty the trash. After a couple of minutes all hell will breaks loose and you will have to click 3 to 4 times on at least 2 clients to get rid of the error messages. That's what we do. I'm sorry, but I can not explain it any other way. It is just not working.
We are testen another mailserver at this moment and are not seeing any problems with IMAP with the same clients. The only thing with that mailserver is, that it laks some features that we really need. So tomorrow we will have to look for another alternative.
Last edited by jimimaseye on 2015-09-02 20:52, edited 1 time in total.
Reason: advertising removed

User avatar
jimimaseye
Moderator
Moderator
Posts: 10053
Joined: 2011-09-08 17:48

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by jimimaseye » 2015-09-02 21:02

Dont get me wrong, Im sure this 'problem' exists (within the Outlook users fraternity) to some degree but given the amount of hmailserver installations there are in use around the world, how come we have just 3 or 4 people mentioning it (to various degrees from "its a deal breaker, I need to look elsewhere" to "it does it regular" to "I rarely see it" to "yeah but its not a deal breaker"). Can it be that companies that have chosen to use Hmailserver instead of Exchange also shun Microsoft as an email client too (opting for alternatives such as Thunderbird etc and therefore there are VERY few Outlook users), or is it that in reality the frequency, or rather proliferation, of the problem is really ...REALLY....minute? (I make no opinion or judgement on this myself on account I am not an Outlook user). Im just interested in peoples thoughts on this.
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

estas
New user
New user
Posts: 7
Joined: 2015-09-02 11:05
Location: Belgium

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by estas » 2015-09-03 08:47

If it was up to us, we would throw away Outlook and use thunderbird. But we do not have that possibility because our crm software can only work with Outlook.
We need a mailserver because our mails are getting filtered by our hoster and we are missing mails this way. We are testing out mailservers together with a temporary domain name where we changed our MX records to point to that server.
I’m very fund of hMailserver because I used it in the past (with POP3) without any problems. But IMAP is a pain in the ass. I cannot explain why we have the problem “all the time” and others never or some times? Our users are not happy with the error messages that popup every time someone else deletes a mail or moves it to another folder.
If time was on our side we would go to Github, download the source codes and fix it ourselves. But for now we only have the option to look for any alternative that can be setup quickly.
It is unfortunately because the hMailserver was configured, worked, I already wrote a vb script to work with our external Blacklist file ect… :(

User avatar
jimimaseye
Moderator
Moderator
Posts: 10053
Joined: 2011-09-08 17:48

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by jimimaseye » 2015-09-03 10:39

Ive just done a little test.

The messageUID within HMS gets changed on a 'per folder' basis rather than a 'per MAILBOX' basis. If I have read it right that RFC says that the messageUID should be retained within the MAILBOX (irrespective of the folder); the uniqueness of the ID must be maintained during the mailbox, whereas HMS seems to maintain ID's within the folder

eg

INBOX
last message ID = 2267
a new message gets 2268

TRASH
last message = 12509
moving the message (UID 2267) from INBOX in to Trash gives it a new ID of 12510.

If you were to move that message out of trash back to Inbox again it would then get messageUID = 2269

So, MAYBE that is the cause. Whether it is right that Outlook should complain about this (when the rest of the worlds email clients dont) is for debate. But it would be SERIOUSLY ironic that a Microsoft product complains when the globally recognised rules (RFC's) DONT get implemented properly (as they have spent a lifetime doing things the way THEY feel they should be done irrespective of the rules and choosing not to fully conform).

Now, why do some people see the problem and some people dont? I dont use Outlook here but I know it does things in a strange way; in particular they use this PST file. Could it be something to do with some use OFFLINE STORAGE and others dont? (Maybe the 'offline' version has a different UID to what it sees it has online) and that those people that do not use OFFLINE storage (filling the PST) doent have the problem?
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
SorenR
Senior user
Senior user
Posts: 6308
Joined: 2006-08-21 15:38
Location: Denmark

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by SorenR » 2015-09-03 10:47

I believe what you are trying to do; multiple clients use one account for storage, is described in RFC 4551.

https://tools.ietf.org/html/rfc4551

It does not look like hMailServer support the extension described in RFC 4551.

https://www.hmailserver.com/documentati ... ernals_rfc
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

estas
New user
New user
Posts: 7
Joined: 2015-09-02 11:05
Location: Belgium

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by estas » 2015-09-03 11:03

mattg wrote:Then tell me what you do, so that we can reproduce it on our servers...
This is ONLY an issue with Outlook using IMAP, and is ONLY an issue at random times, not consistently every time that outlook is used.
FWIW Outlook 2013 has bigger IMAP issues, like only showing some of the available mail.
I'm not willing to give up hMailserver yet, so I did a little test so anyone can reproduce the problem. I also made a clean log file with supported printscreens which I can only provide through PM (if anyone interrested)
I emptied the log file before i started, opened Outlook on 2 clients. Client 1 removes some mails while client 2 does nothing, just waiting in the inbox folder and seeing that the mails are removed indeed. Client 2 goes to the removed items folder and sees the mails that client 1 has just removed. Client 1 right clicks the trash folder and empties this folder. Client 2 sees that the folder is becomming empty. Client 1 removes an other mail from the inbox and BAM client 1 gets 4 errors

I can do this every time, over and over again.

User avatar
jimimaseye
Moderator
Moderator
Posts: 10053
Joined: 2011-09-08 17:48

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by jimimaseye » 2015-09-03 12:01

estas wrote:Client 1 removes some mails while client 2 does nothing, just waiting in the inbox folder and seeing that the mails are removed indeed. Client 2 goes to the removed items folder and sees the mails that client 1 has just removed. Client 1 right clicks the trash folder and empties this folder. Client 2 sees that the folder is becomming empty. Client 1 removes an other mail from the inbox and BAM client 1 gets 4 errors
Client 1 removes to trash, then empties trash, then removes another email...and then has the error complaining of the emails IT has removed no longer having the SAME UID (even though client 1 did the actual deletion??) That sounds more like Outlook not refreshing on its own actions.

Does client 1 have OFFLINE STORAGE enabled or not?

And what happens if you do this exact test again but without client 2 connected?

And also, what happens if Client 1 is running on the SAME machine as the server (out of interest)?
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

chuf
New user
New user
Posts: 11
Joined: 2012-08-15 22:21

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by chuf » 2015-09-03 15:22

Hi All

been using imap with outlook 2007/2010 for the past 3 years and now on 2013 with none of these issues with multiple users. Strange one.......

Chuf

estas
New user
New user
Posts: 7
Joined: 2015-09-02 11:05
Location: Belgium

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by estas » 2015-09-03 15:36

jimimaseye wrote: Does client 1 have OFFLINE STORAGE enabled or not?

And what happens if you do this exact test again but without client 2 connected?

And also, what happens if Client 1 is running on the SAME machine as the server (out of interest)?
Tested both OFFLINE STORAGE enabled and not (same result)
If you do the test again without client 2 there is no problem.
If client 1 is running on the same machine as the server - the problem still exists.

Here is another test that always gets the errors firering: (now with moving mails instead of removing them)
Client 1 and client 2 are both in the inbox folder. Client 1 (or 2 does not matter) moves the oldest mails from the inbox to another imap folder. Client 2 opens that folder. Client 1 tries to move other mails but already the errors occur. If we would leave the oldest mails in the inbox and move recent mails than there is no problem. It has something to do with the UID of the oldest mails.

As for chuf's reply
chuf wrote:been using imap with outlook 2007/2010 for the past 3 years and now on 2013 with none of these issues with multiple users. Strange one.......
multiple users connected to 1 and the same imap account? Because if we would use multiple users and give them all a different imap account than there is no problem!

chuf
New user
New user
Posts: 11
Joined: 2012-08-15 22:21

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by chuf » 2015-09-03 16:05

Setting it up as per your post to see if I can replicate the same......... :shock:
estas wrote:
jimimaseye wrote: Does client 1 have OFFLINE STORAGE enabled or not?

And what happens if you do this exact test again but without client 2 connected?

And also, what happens if Client 1 is running on the SAME machine as the server (out of interest)?
Tested both OFFLINE STORAGE enabled and not (same result)
If you do the test again without client 2 there is no problem.
If client 1 is running on the same machine as the server - the problem still exists.

Here is another test that always gets the errors firering: (now with moving mails instead of removing them)
Client 1 and client 2 are both in the inbox folder. Client 1 (or 2 does not matter) moves the oldest mails from the inbox to another imap folder. Client 2 opens that folder. Client 1 tries to move other mails but already the errors occur. If we would leave the oldest mails in the inbox and move recent mails than there is no problem. It has something to do with the UID of the oldest mails.

As for chuf's reply
chuf wrote:been using imap with outlook 2007/2010 for the past 3 years and now on 2013 with none of these issues with multiple users. Strange one.......
multiple users connected to 1 and the same imap account? Because if we would use multiple users and give them all a different imap account than there is no problem!

chuf
New user
New user
Posts: 11
Joined: 2012-08-15 22:21

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by chuf » 2015-09-03 16:53

Okay
Lets see if ive got this right.......

1 X Imap account on 2 pcs.
Sent a dozen test messages to email account both appeared on both machines in inbox.

Pc1 deleted 6 of the new messages went to trash folder.

Pc2 shows deleted messages in trash folder

Pc1 emptied trash folder delete all.

Pc2 shows empty trash folder.

Pc1 deleted 6 other messages into trash folder.

Pc2 shows 6 new messages in trash folder.

Pc1 emptied trash folder delete all.

Pc2 shows empty trash folder.

Im not getting any error messages? Have I done the same setup as yourself?

Chuf
estas wrote:
jimimaseye wrote: Does client 1 have OFFLINE STORAGE enabled or not?

And what happens if you do this exact test again but without client 2 connected?

And also, what happens if Client 1 is running on the SAME machine as the server (out of interest)?
Tested both OFFLINE STORAGE enabled and not (same result)
If you do the test again without client 2 there is no problem.
If client 1 is running on the same machine as the server - the problem still exists.

Here is another test that always gets the errors firering: (now with moving mails instead of removing them)
Client 1 and client 2 are both in the inbox folder. Client 1 (or 2 does not matter) moves the oldest mails from the inbox to another imap folder. Client 2 opens that folder. Client 1 tries to move other mails but already the errors occur. If we would leave the oldest mails in the inbox and move recent mails than there is no problem. It has something to do with the UID of the oldest mails.

As for chuf's reply
chuf wrote:been using imap with outlook 2007/2010 for the past 3 years and now on 2013 with none of these issues with multiple users. Strange one.......
multiple users connected to 1 and the same imap account? Because if we would use multiple users and give them all a different imap account than there is no problem!

Bill48105
Developer
Developer
Posts: 6192
Joined: 2010-04-24 23:16
Location: Michigan, USA

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by Bill48105 » 2015-09-11 16:04

Hey look i was quoted from year & 1/2 ago :)

Fact this issue is still not resolved is telling. Pain to reproduce & as frustrating as it is for those who run into it, ultra rare. Been forever since I thought about this as none of my 100's of Outlook users hit it even on old custom builds of 5.4. Of course they could have the error & just close it out like the mailbox locked "errors" since it doesn't hurt anything, just annoying.

Cheers all!
hMailServer build LIVE on my servers: 5.4-B2014050402
#hmailserver on FreeNode IRC https://webchat.freenode.net/?channels=#hmailserver
*** ABSENT FROM hMail! Those in IRC know how to find me if urgent. ***

User avatar
jimimaseye
Moderator
Moderator
Posts: 10053
Joined: 2011-09-08 17:48

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by jimimaseye » 2015-09-11 19:01

Bill48105 wrote:Hey look i was quoted from year & 1/2 ago :)

Fact this issue is still not resolved is telling. Pain to reproduce & as frustrating as it is for those who run into it, ultra rare. Been forever since I thought about this as none of my 100's of Outlook users hit it even on old custom builds of 5.4. Of course they could have the error & just close it out like the mailbox locked "errors" since it doesn't hurt anything, just annoying.

Cheers all!
Bill see my post above: viewtopic.php?p=178296#p178296
Does that go towards offering an explanation or helping to identify the problem?
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

thains
New user
New user
Posts: 1
Joined: 2015-09-13 18:19

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by thains » 2015-09-13 19:45

About 20 a year – that’s the amount of mail servers we install every year. Mostly Linux based like Dovecot and others, but sometimes a customer asks for the Windows platform. We came across the IMAP/Outlook problem with hmailserver years ago and that was a major reason NOT to choose hmailserver for our customers.
Because we have (again) a customer that requests a mail server running under Windows, I came across this topic. And I have to say … this makes me angry :x . A user posts a problem that has been there for years and nobody seems to look for an answer. Even worse are the posts some of you do just to prove the other wrong. Shame on you.
If someone is interested. We know of this problem for years and we can simulate it like estas describes in this post. This is the reason why we never use hmailserver in the first place (unless customers only require pop3). We also know many others that have this problem. It’s just too bad that they are not on this forum or any other for what that matters. Maybe than someone will do something about it.
There are several good alternatives if your customers have a low budget or if they do not require many different logins. But that is up to you. Estas if you are interested I will send you some links in a private message.
We normally do not get ourselves mixed in forum discussions, but seeing this topic and the way users try NOT to help others, makes me have a bad feeling.

User avatar
jimimaseye
Moderator
Moderator
Posts: 10053
Joined: 2011-09-08 17:48

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by jimimaseye » 2015-09-13 23:05

Thains....

... you have a VERY wrong idea on this.
thains wrote:A user posts a problem that has been there for years and nobody seems to look for an answer. Even worse are the posts some of you do just to prove the other wrong
You are wrong. First of all, if you read the posts, both this thread, and the other thread, you will see that *I* personally have offered a theoretical explanation (above) of why the workings of HMS may be tormenting Outlook and that Bill had asked for a FOOL PROOF, REPLICATE ON DEMAND method of reproducing the problem. Without it, how can he look for a solution? Simply saying "some of us has the problem" or "*I* can replicate it everytime" doesnt help him if HE cant replicate it with the instructions given (for example, read the thread above and you will see that despite 'estas' giving a recipe for replication, 'chuf' did not get the same result).

And no one here has said anyone is wrong or that the problem doesnt exist. They have said that the method to replicate isnt foolproof, that the number of users suffering it seems to be low and that as an error it doesnt bring the business down (ie, its not critical). And as you have correctly pointed out:
thains wrote:We also know many others that have this problem. It’s just too bad that they are not on this forum or any other for what that matters
if they are not on the forum and telling us about it then either:
a, its no big deal to them or
b, they dont exist.

We can only respond to what is reported.

Now, there is a correct way of reporting such a bug with the application and if you, estas or anyone else wants the true official review by the Developer (Martin) then it should be logged, with reference to a proven example to help, on the Hmailserver Github Issues page. If you do that, then you have a right to be annoyed if it doesnt get fixed. (But at least it will get reviewed.) If, however, they simply pop on to a DISCUSSION FORUM which relies on end-users to offer help saying [sic] "something bad happens", but yet they dont report it on Github, then it simply wont get fixed anyway. And if you only ever sign up to complain about the other non-sufficient reports of the forum not being dealt with in the first place (because they havent been logged on Github correctly), then you definitely will not get a fix either!

There is a hint there, I hope you understand.
(In case you didnt get it: I look forward to seeing your report, with guaranteed method of replication, on github.)
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
mattg
Moderator
Moderator
Posts: 22435
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by mattg » 2015-09-13 23:13

thains wrote: A user posts a problem that has been there for years and nobody seems to look for an answer.
Utter, utter garbage.

We have spent many hundreds of hours chasing this.
This happens sometimes to me. If I remove the Outlook client install that it happens to, and reinstall the issue goes away.
If I don't use outlook, I never get the issue.
Even when I get the message, and have multiple clients connected, I ONLY get this error on Outlook.

I can't force this issue to occur.

I have followed all of the hints and ideas and I cannot replicate this issue from anyone else's directions.

Thanks for TROLLING....
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

estas
New user
New user
Posts: 7
Joined: 2015-09-02 11:05
Location: Belgium

Solved: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by estas » 2015-10-01 10:58

Solved (or sort of) with a workaround.
Our server is up and running and I have to say that it is holding very well – server and clients.
Lots of fine tuning on both server and clients side before it became usable. Many thanks to "thains" who contacted me and solved the problem. It’s not a complete solution rather a workaround, but it is working and we are happy. :D

User avatar
jimimaseye
Moderator
Moderator
Posts: 10053
Joined: 2011-09-08 17:48

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by jimimaseye » 2015-10-01 11:55

Good to hear, Would you be able to share for other users?

(This is particularly interesting, especially as he said:
thains wrote:This is the reason why we never use hmailserver in the first place
Kind of makes you wonder how he know of a workround if, as he says above, he doesnt use it in the first place.)
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

Bill48105
Developer
Developer
Posts: 6192
Joined: 2010-04-24 23:16
Location: Michigan, USA

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by Bill48105 » 2015-10-06 05:15

Sorry I didn't get notifications of responses or I missed them. Yeah jimimaseye that is interesting & supports what we suspected for awhile. The message UID's get out of order or at least in terms of what Outlook is expecting. That's the key is why is it that only Outlook has the issue & why only sometimes for some people. Granted usage patterns of email vary (heck I NEVER delete emails where some people might delete one at a time as they read & others might mass delete etc) And I've tried doing various things like moving mail around, deleting, undeleting etc but heck if I could get it to happen after more hours than I can count working on it. Heck I setup automated tools to put heavy load on test servers delivering MILLIONS of emails a day without it happening. Thanks for sticking up for us btw. It's not fair for someone to claim we've not tried to help especially considering the 1000's of hours people volunteer to support fellow hmailserver users for free. Heck mattg alone has helped out more people than the population of a small city lol
Bill
hMailServer build LIVE on my servers: 5.4-B2014050402
#hmailserver on FreeNode IRC https://webchat.freenode.net/?channels=#hmailserver
*** ABSENT FROM hMail! Those in IRC know how to find me if urgent. ***

User avatar
mattg
Moderator
Moderator
Posts: 22435
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by mattg » 2015-10-06 05:56

Bill48105 wrote:Heck mattg alone has helped out more people than the population of a small city lol
G'day Bill,
Good to see of you again.

Yep more than triple the number of posts then my home town population. :)
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

Bill48105
Developer
Developer
Posts: 6192
Joined: 2010-04-24 23:16
Location: Michigan, USA

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by Bill48105 » 2015-10-06 17:26

mattg wrote:
Bill48105 wrote:Heck mattg alone has helped out more people than the population of a small city lol
G'day Bill,
Good to see of you again.

Yep more than triple the number of posts then my home town population. :)
Good day to you good sir!
LOL nice I called that! You must be in small town but hey you also post too much too :D
hMailServer build LIVE on my servers: 5.4-B2014050402
#hmailserver on FreeNode IRC https://webchat.freenode.net/?channels=#hmailserver
*** ABSENT FROM hMail! Those in IRC know how to find me if urgent. ***

User avatar
mattg
Moderator
Moderator
Posts: 22435
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by mattg » 2015-10-06 17:36

Nah mate, I'm from a big town.

This is a little town just down the road a bit >> http://nindigully.com/about/
Today the boom town of approximately 50 people and 15 houses of the early 1900s have been reduced to two houses, the pub, the old General Store and the Town Hall with a population of six!
That's SIX people. There's more snakes.
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
SorenR
Senior user
Senior user
Posts: 6308
Joined: 2006-08-21 15:38
Location: Denmark

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by SorenR » 2015-10-06 18:38

mattg wrote:Nah mate, I'm from a big town.

This is a little town just down the road a bit >> http://nindigully.com/about/
Today the boom town of approximately 50 people and 15 houses of the early 1900s have been reduced to two houses, the pub, the old General Store and the Town Hall with a population of six!
That's SIX people. There's more snakes.
Likewise you also have the biggest farms...

http://www.news.com.au/finance/real-est ... 7420251801

http://www.dailymail.co.uk/news/article ... alist.html

The combined territory of the properties is 101,411 square kilometres, or almost 11 million hectares, equivalent to the US state of Indiana, and more than three-quarters the size of England.
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

User avatar
Dravion
Senior user
Senior user
Posts: 2071
Joined: 2015-09-26 11:50
Location: Germany
Contact:

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by Dravion » 2015-10-07 00:39

Hmm this problem looks to me like an transaction isolation problem (like it can happen in sql) or a late event notification situation for the 2nd IMAP-Client.Number 2 thinks he is uptodate but isnt and references a null object or an array element which is allready freed or reallocated by the memory management function.

Maybe we need a small imap console debug client wich can create multiple (overlapped) IMAP-connects to the same IMAP-Inbox triggering simultaenously delete operations.

User avatar
Dravion
Senior user
Senior user
Posts: 2071
Joined: 2015-09-26 11:50
Location: Germany
Contact:

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by Dravion » 2015-10-08 06:27

Howdy. I have some news on this one but no good ones. This is indeed an long term Outlook Problem since atleast Version 2003. Almost every other Imap server project outthere (opensource or not) is confronted with exactly the same uid error problem. Mailtrac claims (very unspecific) newer versions of its imapd can handle this problem. The second imap-server (Zimbra) a Developer on bugtracker with Name Flennigan describes this problem as an Outlook uid caching Problem. All other major projects like Dovecot imapd, courier imapd, cyrus imapd, university of washington uw-imapd, roundcube and communigate ect having lots of Outlook users with exact the same problem - but - again - no solution at all. ONLY Mailtrac claims it has a fix for it in its commercial, closed source, payware product, but i dont know if this is the truth. Even AOL and Yahoo Outlook-imap users reporting this UID issue and Microsoft support is reporting "we know the problem but a solution is not avaiable".

User avatar
Dravion
Senior user
Senior user
Posts: 2071
Joined: 2015-09-26 11:50
Location: Germany
Contact:

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by Dravion » 2015-10-08 06:41

This is Mailtraqs statement at the UID Problem:

Users of older versions of Mailtraq should upgrade to the latest version (2.16.3.3022 or above) where an update to the Mailtraq IMAP service prevents message list reloads in response to asynchronous change notifications which mitigates this Outlook behavior

User avatar
mattg
Moderator
Moderator
Posts: 22435
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by mattg » 2015-10-08 07:51

Thanks for the research Dravion
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
Dravion
Senior user
Senior user
Posts: 2071
Joined: 2015-09-26 11:50
Location: Germany
Contact:

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by Dravion » 2015-10-08 08:32

No Problem :D

hMailserver is verry cool project and its worth the effort. The community is active and helpfull in most situations.

Regarding the IMAP/Outlook Problem. Maybe we should concentrate on implementing our own MAPI Message Store Provider (Outlook native Interface) and we get more Performance for Outlook users and a taste of MS-Exchange (Public Folders, Central calender, Central Adressbook and Central Tasks) :D

User avatar
jimimaseye
Moderator
Moderator
Posts: 10053
Joined: 2011-09-08 17:48

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by jimimaseye » 2015-10-08 09:37

I hope Terry Troll (quoted here) is reading your update Dravion.
thains wrote:About 20 a year – that’s the amount of mail servers we install every year. Mostly Linux based like Dovecot and others, but sometimes a customer asks for the Windows platform. We came across the IMAP/Outlook problem with hmailserver years ago and that was a major reason NOT to choose hmailserver for our customers.
Because we have (again) a customer that requests a mail server running under Windows, I came across this topic. And I have to say … this makes me angry :x . A user posts a problem that has been there for years and nobody seems to look for an answer....
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
Dravion
Senior user
Senior user
Posts: 2071
Joined: 2015-09-26 11:50
Location: Germany
Contact:

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by Dravion » 2015-10-08 10:07

Yeah its strange to see thadt Microsoft doesnt bother about this flaw in one of its Major Office Programms.
This flaw must exists (at least 10 years, ubelievable). In the MS Support site there was feedback window and i should provide some textfeedback.
Well my feedback was: Hurry up, you lazy Bastard and fix this Error asap!!! :)

estas
New user
New user
Posts: 7
Joined: 2015-09-02 11:05
Location: Belgium

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by estas » 2015-10-20 14:28

Hello everyone, I was hoping that “thains” would explain his solution to this problem but I guess that he is still a little off by the name calling. It is not nice to call somebody a troll if you do not know the person. I only know this guy for a couple of weeks and I have learned that he is well respected in the IT world and I’m very great full that he came to the rescue.
So here is what we did to fix this problem with Outlook (it’s not a complete fix, in some cases, not often, the UID error still pops up)

First on server side (IMAP folders): Instead of using the account folders (account->advanced->edit folders), we used the IMAP->Public folders. We noticed that there is a difference in the way Outlook uses these folders. If many people use the same login account, there was an issue with the folders Outlook standard pushes into the IMAP account. (Folders like spam, trash, send items, concepts) Best practice is to use the public folders when multiple users login to the same account.
We use the public folders to automatically filter all spam, newsletters, mails from social media and a special folder where mails are stored once they have passed our CRM attached rule. Be aware NOT to use the same names for these folders as Outlook’s. So we renamed the SPAM folder to “Canned meat”. Together with some scripting all the spam and commercial mails are automatically filtered into the public folders (more than 50% of the mails are spam). You can do great stuff with the scripting feature.

Second on Outlook side: In your account setting (settings for internet e-mail) -> deleted items (for Outlook 2010) or Advanced (for Outlook 2013) there is a setting called “mark items for deletion” (hope this is correct English because my Outlook is in Dutch). Check this but uncheck the next item called “Delete items when switching folders”. We also played with the advanced setting “time-outs for server” but I’m not seeing things improve with this item.
In Outlook File->Options->advanced->send/receive->edit you can uncheck the option not to count the unread folders. Probably helps but we have already undone this because it is handy to have.
We have 2 teams (2 accounts) and all users from the same team log in with the same account. One team has permissions in all public folders, the other team only have reading rights and some folders are even hidden. This way everybody benefits from the IMAP functionality, and most of the users (team 2) never see mails being moved from one folder to the other. They will never get the UID error, team 1, who does all the work attaching mails into our CRM software and moving mails into the correct folder, have mentioned 2 occasions in 2 weeks time.

This is the way we handled our UID problem.
I’m still a very big fan of hMailserver, I love the community and the fact that so many people are participating.
Hope this will help others.
Peace and happiness…

User avatar
jimimaseye
Moderator
Moderator
Posts: 10053
Joined: 2011-09-08 17:48

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by jimimaseye » 2015-10-21 00:16

Hi Estas.

Compliments and thanks to you on posting the partial workround for the benefit of other users and for attempting to stick up for Thains and try to moderate between him and the attention he has drawn.

It has been made clear about where the problem lies by more than one person and also shown that Hmailserver is not necessarily the problem. For what its worth, I find it still a little strange and unfortunate that the "workround" involves so much work/change and even after that it still isnt fool proof.

Regarding Thains: It may well be that he is a well respected I.T. guy (to those that know him) but this does not excuse him or make him any different from other people (known as 'Trolls' in Internet forum speak) that simply barge in, shout their mouth off for no other reason than to antagonise other people and then leave. He made several comments that apart from being untrue were actually out of order to suggest and could have been offensive to other posters; namely those people that HAVE 'listened' and done work to try and resolve this issue (including me, Matt, Bill, chuf and others -ALL posting and investigating well before he stepped in). I even pointed out to him his errors and how to correct his reporting in order to get a more reactive and appropriate resolve in the manner he would like but he simply wanted to just dump his bile over everyone else's work and then exit quickly.

Respected I.T. guys are only respected when their knowledge is shared and benefit others, and we relish them when they help out here in this forum. But until he does that, and meanwhile continues to GOB OFF upsetting others with his inane, confrontational, pointless nonsensical 'contributions', he will just be an rude unrequired 'participant'. (See http://www.collinsdictionary.com/dictio ... lish/troll, version 6, or https://en.wikipedia.org/wiki/Internet_troll - the opening paragraph, and bear it in mind when you read his post in context with all others. Then you will see that Matt was right. Calling a man a troll "when you do not know the person" is wrong, I agree, but calling a man a troll when ALL he has done is give 100% trolling material is correct.

We are glad for your help, but personally I really dont want his idea of help. For everything he has offered he can stay well away.

(Sorry, I know it was a little inappropriate to use this thread and give such a reply but I feel really strongly about this type of interference and I needed to make it clear).
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

JacGriffin
New user
New user
Posts: 1
Joined: 2015-11-19 09:09

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by JacGriffin » 2015-11-19 09:44

I cannot explain why we have the problem “all the time” and others never or some times? Our users are not happy with the error messages that popup every time someone else deletes a mail or moves it to another folder.

User avatar
mattg
Moderator
Moderator
Posts: 22435
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by mattg » 2015-11-19 09:50

Does this happen with public IMAP folders, or just for when multiple users log onto the same IMAP account and move / delete stuff

Do the users experience this issue with other mail clients, eg Thunderbird?
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
Dravion
Senior user
Senior user
Posts: 2071
Joined: 2015-09-26 11:50
Location: Germany
Contact:

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by Dravion » 2015-11-19 13:29

Hmm. maybe you should consider a redesign of your workflow which doesnt require dual use of
imap folders in the first place. As stated before, there are many imap server out there with exactly the
same problems.

Some ideas:
*Write a hMailserver VBScript for message delegations to specific inboxes.
*Create a mailinglist for tean mails (for example: support-list@your-company.com)

Anyhow:
It would be a great help if you can setup a hMailserver Installation+Mailclient(s) in Virtualbox
http://download.virtualbox.org/virtualb ... 61-Win.exe trigger the Error and "pause" the VM at the error popup.You should then zip your VM folder and upload it to dropbox/skydrive/googledrive location so we can download it and take a look at the live error situation so we can investigate the problem on the tcp packet layer.

silc
New user
New user
Posts: 9
Joined: 2012-09-29 11:07

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by silc » 2016-06-22 17:40

Same problem with 5.6.5 beta...

User avatar
mattg
Moderator
Moderator
Posts: 22435
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by mattg » 2016-06-22 23:53

Yes I'm sure that it is the case.

We have NEVER been able to find a cause that is reproducible on another system.
If you can help with that, then we can fix this issue.

Been hanging around for many many years, I get it myself at times, but there is no rhyme nor reason that we can establish. One day my client computer will just get error after error after error, and the next day the exact same computer will be fine...
Perhaps this is related to the size of the emails...

It always seemed to affect me and no-one else at any of the workplaces I'd visit.
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

kbm
New user
New user
Posts: 10
Joined: 2016-06-22 23:52

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by kbm » 2016-06-24 14:53

I often (but not always) see this error when moving messages from the inbox to another folder via the native android mail app. When i return to my computer (outlook has been running all this time) the UID error is front and centre on the screen

There doesn't seem to be any negative behaviour to speak of.... just the error message. 1 mouse click and its gone

hmbaer
New user
New user
Posts: 20
Joined: 2019-05-01 14:29

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by hmbaer » 2019-05-01 14:47

Is this problem solved?

If not, will it only occur with Outlook?

Thank you.

User avatar
Dravion
Senior user
Senior user
Posts: 2071
Joined: 2015-09-26 11:50
Location: Germany
Contact:

Re: IMAP - outlook (2010) UID issue still not solved with version 5.6.4 B2283

Post by Dravion » 2019-05-01 14:52

As far as we know, this is a Outlook only Problem.
But it seems in mondern Outlook versions it occurs far less.

Post Reply