Reasons for NOT upgrading...

[jimimaseye]

Ok, I thought it was time for me to ask out there to our learned (and obviously more experienced) advisors, friends and users to offer me and other readers some advice and opinion.

I am currently running production release HMS 5.4.2 B1964 running on Mysql 5.2

Having been monitoring the forums for some months I decided for my own benefit that I am happy to stay where I am and not to upgrade to the latest 5.6.1. The reasons for this are:

I see many posts seemingly fo the same problems that are a consequence of upgrading to 5.6.1. From memory (and this is the important bit, it is from MEMORY) it includes:

1)
a, There is some issue about some encoding incompatibility which HMS 5.6.1 addresses but yet is not compatible with MySQL 5.2 - requiring v5.5 if using HMS 5.6.1 is to be used.
b, After upgrading, there are often 'authentication errors' - this is something to do with default implementation of SSL/StartTLS set somewhere. (I have no SSL certification in use and use unencrypted standard authentication/comms.)
c, Upgrading has been problematic to some people (ie BACKUP and then RESTORE of data)
d, The FORWARDING feature has been changed to automatically change the FROM address from the original sender to the local forwarders addresses (resulting on an end user seemingly receiving an email that looks like it has been SENT by my company employee (their email) when the reality it was sent by someone external)

2) I dont think I am suffering any issues that need attention or fixing that has been dealt with in newer versions ("If it aint broke, then dont fix it" approach)

Now, I know that these things are addressed and rectifiable by means of

1a, doing the upgrade if you need to
1b, understanding WHERE the settings are set and how to change them,
1c, not using the internal backup although evidence fro some users still shows the restore having a problem
1d, use a script to override it

and

2, do this post and get opinion back to find out if I am right to do so.

But SHOULD I BE CONCERNED about the points above? Remember I said these things are from MEMORY and have deliberately not referred to any one thread as an example (but I KNOW they are there).

SO I would like to ask out expert friends if they could sanitise my concerns (ie, how right or wrong am I having these concerns in the form I have written) and to offer their concise response (and solution if necessary) to those concerns. (Im sure they understand the detail in the points I am talking about above). And ultimately an opinion to the thread question: is there REALLY a reasons not to upgrade, or rather, is there REALLY a reason and benefit to upgrade and what are those main benefits?

(I have made some of the points simplified for the benefit of other readers)

Discuss.

[martin]

1A) Are you referring to the UTF8 message indexing? If so, nothing gets worse by upgrading. You'll have MySQL errors in the old version as well, but in the new version its possible by work around it by changing encoding manually in the database.
1B/C) Can you tell me more about this one?
1D) This behavior is reverted in 5.6.2.

If everything is working, there's not really any reasons to upgrade - unless there are security fixes in later releases which you want to have. 5.6.1 fixed the following, which could be of interest if you're relying on PHPWebAdmin and have server administrators created on domain level:

Issue 77: Domain administrators should not be able to change server administrator password (security)

[SorenR]

1)
a, There is some issue about some encoding incompatibility which HMS 5.6.1 addresses but yet is not compatible with MySQL 5.2 - requiring v5.5 if using HMS 5.6.1 is to be used.

Yeah, those pesky EMOJI's from Twitter and god knows who else have an iPhone

I fixed the 4 byte UTF-8 EMOJI problem this way:

Code: Select all

   Sub FixEmojis(oMessage)
      Dim strValue, strEmoji, ECFlag, SFlag : SFlag = False
      ECFlag = oMessage.EncodeFields
      oMessage.EncodeFields = False
      If (InStr(oMessage.Subject, "=F0=9F=") > 0) Then
         strEmoji = Mid(oMessage.Subject, InStr(oMessage.Subject, "=F0=9F="), 12)
         strValue = Replace(oMessage.Subject, strEmoji, "")
         oMessage.Subject = strValue
         SFlag = True
      End If
      If SFlag Then oMessage.Save
      oMessage.EncodeFields = ECFlag
   End Sub

[mattg]

1)
a, There is some issue about some encoding incompatibility which HMS 5.6.1 addresses but yet is not compatible with MySQL 5.2 - requiring v5.5 if using HMS 5.6.1 is to be used.
b, After upgrading, there are often 'authentication errors' - this is something to do with default implementation of SSL/StartTLS set somewhere. (I have no SSL certification in use and use unencrypted standard authentication/comms.)
c, Upgrading has been problematic to some people (ie BACKUP and then RESTORE of data)
d, The FORWARDING feature has been changed to automatically change the FROM address from the original sender to the local forwarders addresses (resulting on an end user seemingly receiving an email that looks like it has been SENT by my company employee (their email) when the reality it was sent by someone external)

1B/C) Can you tell me more about this one?

1.b
SOME (very few) users have reported issues when their ISP uses a self signed or poorly established certificate, and they connect via POP3 External download, or send via SMTP relayer. De-Selecting the 'Verify remote server SSL/TSL certifiactes' fixes the symptoms, but of course in these cases the problem is the other end.

I have this selected now without issue for many months

1.c
We have seen two or maybe three people who claim that the built in backup and restore hasn't worked for them.
I suspect that these have been caused by user error, not program error.

There was an issue a while back that may still exist, in that the 'uncompressed' backups don't back up messages. You had to do a compressed backup to backup messages.I have outgrown the builtin backup so I can't test this. I know that a robocopy of the datadirectory, eventhadlers.vbs, and hmailserver.ini file, and a MyQL backup / restore of the database, followed by a restart is enough for me to backup and restore (tested this week on 5.6.1 - have since upgraded to 5.6.2) I'll be able to keep a very recent backup automatically ready for hot swap very soon.

2) I don't think I am suffering any issues that need attention or fixing that has been dealt with in newer versions ("If it aint broke, then dont fix it" approach)

StartTLS
Never thought that I would use it much, but awesome. Love it.
Outlook and iDevice mail accounts very happy now (to name a couple).

It also surprise me how much inter-server StartTLS traffic there actually is. I'd guess that far more than half of my server to server traffic is StartTLS encrypted, possibly close to 3/4. (I should script some logging to see exactly how much)


ICOP errors.
If you don't get them - great. If/when you get them, you will need to upgrade

SSL/TLS cyphers
Some of the cypher stuff, depends on your corporate need.

Yeah, those pesky EMOJI's from Twitter and god knows who else have an iPhone

They also come from Facebook, and a heap of other places too.

If you have teenagers who use your server, you will need this fix soon enough.

Matt

[jimimaseye]

Thanks for the replies chaps.

Martin, I left Matt to answer the question to references (thanks Matt) as he was more best placed to answer the questions (he sees and understands more than me - hence why *I* am the one asking the question )

1.c
We have seen two or maybe three people who claim that the built in backup and restore hasn't worked for them
I suspect that these have been caused by user error, not program error.

Example: viewtopic.php?p=173112#p173112).

Heres the thing: these problems are still existing for these people as far as we know. And I have 2 problems with it:
1, there is a suggestion that 16gb in data size is "too big" for the internal backup yet (eg, see : viewtopic.php?p=173468#p173468) quote, "Once the builtin backup stops working (at 16 Gb!!)..." as far as I can see the backup does nothing more than what we do externally (copying data directory, and zipping it) as well as writing out the message details to the XML file. I dont understand why the has to be a limit suggested.
2, IF there is a limit, and IF you do exceed this limit, how the squidly is someone supposed to upgrade MySQL database version? By suggestion, the theory is:
a, backup using INTERNAL HMS backup procedure
b, Upgrade the MySQL database version
c, Restore from backup using internal procedure

But you cant do the initial backup if the original database size was too big.

In any case, going back to the quote, I struggle to understand (yet to be concluded though) how you can do "user error" in a single 1 click backup/restore procedure. (again we are yet to determine the conclusion to this). So it needs to be proven that there is DEFINITELY no problem with the backup/restore procedure in these later versions

UTF8: dont use indexing so shouldnt have a problem currently. Ok.

StartTLS
Never thought that I would use it much, but awesome. Love it.
Outlook and iDevice mail accounts very happy now (to name a couple).

So I infer from that is that it just helped with a couple of clients in connecting. Other than this (because as far as I am aware I am not suffering any problems - we use Thunderbird desktop client and iPhone's stock client which DONT have a problem with current authentication methods (is this a surprise?), are there other benefits and considerations? It simply helps to mail servers to talk to each other encrypted providing a tad more security (And this works WITHOUT me having any form of certificate on our mailserver?)

[mattg]

And this works WITHOUT me having any form of certificate on our mailserver?

You need a certificate for connections made TO your server.

You can still connect to other servers via StartTL without a cert.

how the squidly is someone supposed to upgrade MySQL database version?

Easy enough to backup MySQL and then Restore MySQL data to new version

Don't need to do a special hMailserver backup, just backup the MySQL database(s).

[jimimaseye]

So for people upgrading the database its a case of:

mysqldump to backup (as per the current nightly backup) (In my case currently:

Code: Select all

mysqldump -u"root" -p"thatwouldbetelling" -q -A -l --add-drop-table -P3306 > backup.sql"

and then the

Code: Select all

mysql < backup.sql

to recover? is that about right? This should CREATE a database from scratch or is it assuming the hmailsrver database still exists (as it did before the program MySQL version update was ran)?

or, its more like:

Code: Select all

mysql> CREATE DATABASE IF NOT EXISTS mycompanydb1;
mysql> USE mycompanydb1;
mysql> source backup.sql

(Ooh, I have just found out my MySQL version is already 5.5 )

Just to know so I can maintain and also advise others if need be.

[mattg]

or, its more like:

Code: Select all

mysql> CREATE DATABASE IF NOT EXISTS mycompanydb1;
mysql> USE mycompanydb1;
mysql> source backup.sql

yep, it does that

[prisma]

c, Upgrading has been problematic to some people (ie BACKUP and then RESTORE of data)

The only thing I'm able to remember regarding backup was a problem with restoring uncompressed backups:
viewtopic.php?t=25571
But this is definitely not related to 5.6 and has always been that way.

Please keep this thread updated regarding restore issues. Following Martin I'm also very interested in it.


(*) Bill told me, this behaviour was "by design", but it's hard to imagine...

[jimimaseye]

I find that interesting to read and actually backs up a theory I had as to why people are having problems (it was only a theory to me, though, because I based it on doubt and had not actually tried it).

I must say I remain divided about the idea of it clearing out the data directory regardless of whether the restore involves data restore or not (I can see and understand both points of view as to why and why not). BUT I simply cant understand how or why it ONLY works as long as it is starting from a compressed file.... given that the initial backup offers you the CHOICE to compress or not. (At no point in the backup process does it say "you can compress or not compress but remember if you dont compress you cant restore your data despite you thinking you have backed it up"). My idea would be that if you state "backup message" (and those messages are indeed placed in a backup folder) then they would still be there for restoring from therefor making it irrespective of your choice to compress or not.

Anyway, I guess I should reserve this post for the thread posted above and for future help advice for users. But hopefully it raises the issue to Martin (as requested)

[mattg]

(*) Bill told me, this behaviour was "by design", but it's hard to imagine...

+1

[jimimaseye]

Back to initial topic: I have just upgraded. Now I am on 5.6.4 B2283.

My only dislike (so far) of 5.6.4 is that the default logging of DNSBL which appeared as 'APPLICATION' logging in 5.4.2 is removed to 'TCPIP' logging only (so therefore I can no longer see it - and I dont want to enable TCPIP logging because of all the other unnecessary bumph it logs). There is also a little bug in that although the DNSBL lookups appear in TCPIP, the SURBL lookups do not and you have to enable 'DEBUG' to see these - a definite inconsistency and I see no reason or explanation for it.

It's a pain because enabling TCPIP you get a lot of rubbish listed telling you nothing (especially if your emails come in by External Download, as mine does) and enabling DEBUG is just plain ridiculous in the amount of unnecessary information it gives for normal day-to-day running. Its infeasible to take on stupidly large log files of TCPIP and DEBUG, full of nothing important, just to be able to see the "MATCH: YES or NO" DNSBL/SURBL lookup results. I found 'APPLICATION' (with its inclusion of the lookup matches) + SMTP sufficient to provide meaningful and useful log files for day-to-day running. With the upgrade I now go to receiving either even LESS information in my logs than I did before (if I keep APPLICATION and SMTP), or a stupid amount more just to see what I used to see before.

Ho hum.

I dont have any certificates so I dont have problems with implementing them and as far as I can see everything else is working fine. (I might even go as far to say as the Admin seems a little bit quicker/responsive. Could it be true or am I just wishful thinking?)