I am alive

Use this forum if you want to discuss a problem or ask a question related to a hMailServer beta release.
Post Reply
User avatar
martin
Developer
Developer
Posts: 6837
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

I am alive

Post by martin » 2011-07-25 00:49

I know it was quite some time since I was active around here. I also know some of you are waiting for a stable version 5.4. There are a few things (mainly small stuff) which needs to be fixed before we can release 5.4 as stable. I will start to work on that now.

For instance, one thing which is remaining is that the backup/restore functionality does not work properly now when we no longer store the full paths in the database.

Martin

Bill48105
Developer
Developer
Posts: 6192
Joined: 2010-04-24 23:16
Location: Michigan, USA

Re: I am alive

Post by Bill48105 » 2011-07-25 01:44

Hey Martin,
Welcome back! Yeah the inability to open messages in queue should be fixed too. (Work-around is to modify the shortcut to 'Start in' the Data folder but I'd imagine the Data folder should be pre-pended to the file path in the GUI)

As far as backup/restore, wow it'd be great to fix it so it works on bigger data sets as the ~1GB limit isn't ideal but I haven't even looked at the code to see if there would be an easy fix or if it needs to wait til 5.4.1 to avoid delaying 5.4 for now. Maybe I'll dig into that tonight as now I'm curious & had been meaning to look but keep getting sidetracked.

Yell if there are any of those things you want me to work on so we're not duplicating efforts.

But WOO WOO! is what I say! :D
Bill
hMailServer build LIVE on my servers: 5.4-B2014050402
#hmailserver on FreeNode IRC https://webchat.freenode.net/?channels=#hmailserver
*** ABSENT FROM hMail! Those in IRC know how to find me if urgent. ***

User avatar
mattg
Moderator
Moderator
Posts: 21257
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: I am alive

Post by mattg » 2011-07-25 02:59

Welcome back Martin.

Hope all is good in your life etc...

Matt
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

bescher
Normal user
Normal user
Posts: 123
Joined: 2008-05-26 01:56
Location: Milwaukee Wi
Contact:

Re: I am alive

Post by bescher » 2011-07-25 10:15

Welcome Back Martin Great to see you. I agree with Bill on the backup/restore if it can be larger 2 gig or even 2.5 would be fantastic.

Now all we need is Nico to pop up with a newer version of his Clamav and the Clamav extra databases

User avatar
martin
Developer
Developer
Posts: 6837
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: I am alive

Post by martin » 2011-07-25 13:09

The problem with the 2GB backup limitation is that hMailServer uses a ZIP library incapable of larger files than 2GB. I think that the backup/restore functionality needs to be revamped a bit. It's slow and it's not transactional which may cause confusion. For instance, when restoring a backup a 'ghost' message may appear (same message in two folders), even if it's unlikely.

Bill48105
Developer
Developer
Posts: 6192
Joined: 2010-04-24 23:16
Location: Michigan, USA

Re: I am alive

Post by Bill48105 » 2011-07-25 14:54

Think 7z lib is avail but need to confirm compatible with hmail's licensing. Didn't want to sidetrack getting 5.4 released this could very well wait until 5.4.1 if needed, just mentioning something that seems to come up quite a bit that would make hmail immensely more user-friendly not only for backing up but for migrating between servers etc.

As I said let me know which parts if any you want me to work on to get this thing moved forward.
Thx!
Bill
hMailServer build LIVE on my servers: 5.4-B2014050402
#hmailserver on FreeNode IRC https://webchat.freenode.net/?channels=#hmailserver
*** ABSENT FROM hMail! Those in IRC know how to find me if urgent. ***

User avatar
katip
Senior user
Senior user
Posts: 804
Joined: 2006-12-22 07:58
Location: Istanbul

Re: I am alive

Post by katip » 2011-07-25 19:04

bescher wrote:Welcome Back Martin Great to see you. I agree with Bill on the backup/restore if it can be larger 2 gig or even 2.5 would be fantastic.

Now all we need is Nico to pop up with a newer version of his Clamav and the Clamav extra databases
+1 (Nicooo!)

Hi Martin, good to see you again.
Katip
--
HMS 5.7.0 x64, MariaDB 10.4.10 x64, SA 3.4.2, ClamAV 0.101.2 + SaneS

Bill48105
Developer
Developer
Posts: 6192
Joined: 2010-04-24 23:16
Location: Michigan, USA

Re: I am alive

Post by Bill48105 » 2011-07-25 19:15

Am I missing something? Do people think martin has special "Bat phone" link to Nico or something? LOL

Looking at the GUI code it looks like there is no (obvious) reason why there is a 1.5G limit on backup as long as the user does not check 'compress' because all that happens in that case is a DIR copy is made. Should be possible to do external compressor or even 7z though.

Again IMO top priority should be to get final touches done to get 5.4 out finally so don't want to distract from that. The comment about backup/restore had just reminded me of the hassles of the the 1.5G limit for many people. Obviously the path stuff needs to be fixed either way.
Bill
hMailServer build LIVE on my servers: 5.4-B2014050402
#hmailserver on FreeNode IRC https://webchat.freenode.net/?channels=#hmailserver
*** ABSENT FROM hMail! Those in IRC know how to find me if urgent. ***

rjk
Normal user
Normal user
Posts: 248
Joined: 2010-03-30 19:30
Location: uʍop ǝpısdn

Re: I am alive

Post by rjk » 2011-07-25 19:41

Good to hear from you, Martin!
I think 7z would be a useful addition to 5.4.1 :)
hMailServer 5.5.2-B2129 on Server 2008 R2 VM
MySQL 5.5.25, IIS 7.5, PHP 5.6.2 via FastCGI, RoundCube 1.0.3
XenServer 6.0 on HP DL380 G5 32GB RAM

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: I am alive

Post by ^DooM^ » 2011-07-25 19:45

7Zip has my vote too.
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

User avatar
mattg
Moderator
Moderator
Posts: 21257
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: I am alive

Post by mattg » 2011-07-26 08:12

^DooM^ wrote:7Zip has my vote too.
Yep, me too
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

bescher
Normal user
Normal user
Posts: 123
Joined: 2008-05-26 01:56
Location: Milwaukee Wi
Contact:

Re: I am alive

Post by bescher » 2011-07-26 11:25

No Bill I do not think Martin has a direct link to Nico. I was just mentioning another part of my dream list

prisma
Senior user
Senior user
Posts: 310
Joined: 2010-07-09 13:16

Re: I am alive

Post by prisma » 2011-07-28 09:58

mattg wrote:
^DooM^ wrote:7Zip has my vote too.
Yep, me too
Yep, me too. An improvement of the backup seems to be necessary and would be good reason to wait some more weeks. I like the build in backup very much, but we have nearly reached the 2gig limitation. So, I'd be the first who likes to test it !! A good reason to update :)

Welcome back Martin!

Bill48105
Developer
Developer
Posts: 6192
Joined: 2010-04-24 23:16
Location: Michigan, USA

Re: I am alive

Post by Bill48105 » 2011-07-28 15:46

prisma wrote:Yep, me too. An improvement of the backup seems to be necessary and would be good reason to wait some more weeks. I like the build in backup very much, but we have nearly reached the 2gig limitation. So, I'd be the first who likes to test it !! A good reason to update :)

Welcome back Martin!
FYI: I have tested & found a bug in the backup system that does not stop it at 1.5G for mssql ce as expected. In doing so it appears I have confirmed what I believed was true: If user does not choose compress backup then there really is no real reason to restrict to 1.5G. Am thinking of making the 1.5G limit only enforced if the user chooses compress for now while fixing the bug. That would allow people who have > 1.5G to still use internal backup as long as they don't compress (zip) it. Then they can compress the backup folder using whatever tool they choose after backup completes. That'll let us not hold up 5.4 & maybe try to get 7z in for 5.4.1. ;)
hMailServer build LIVE on my servers: 5.4-B2014050402
#hmailserver on FreeNode IRC https://webchat.freenode.net/?channels=#hmailserver
*** ABSENT FROM hMail! Those in IRC know how to find me if urgent. ***

User avatar
Slug
Moderator
Moderator
Posts: 1369
Joined: 2005-03-13 05:42
Location: Sydney Australia
Contact:

Re: I am alive

Post by Slug » 2011-07-28 15:50

bescher wrote:No Bill I do not think Martin has a direct link to Nico. I was just mentioning another part of my dream list

I have emailed Nico a few times now with zero response, not sure if he is still alive :shock: I hope so.

And 7zip has my vote as well ..

Michael
Missing Hmailserver ... Now running Debian servers

User avatar
martin
Developer
Developer
Posts: 6837
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: I am alive

Post by martin » 2011-07-28 17:49

I've fixed the problem with viewing messages in Administrator. Currently creating a new build.

Regarding backup compression. A couple of months ago, I spent some time trying to find an open source alternative which I could use in hMailServer. 7zip was one of the alternatives I looked into. I can't recall exactly what the problem was, but while it was open source, I came to the conclusion that i had to use some part of 7zip which was not compatible with the hMailServer licensing. But when I look into it now, the 7zip SDK seems to be public domain which means that it shouldn't be a problem. To make it really simple, hMailServer could just launch the 7zip executable and perform the compression, rather than trying to integrate the 7zip engine into hMailServer. I'll contact the 7zip author and check.

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: I am alive

Post by ^DooM^ » 2011-07-28 20:08

If that doesn't pan out martin just allow us to enter a string to be ran after backup so we can use whatever compression program we want in much the same way the AV works.
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

Bill48105
Developer
Developer
Posts: 6192
Joined: 2010-04-24 23:16
Location: Michigan, USA

Re: I am alive

Post by Bill48105 » 2011-07-28 22:54

^DooM^ wrote:If that doesn't pan out martin just allow us to enter a string to be ran after backup so we can use whatever compression program we want in much the same way the AV works.
Or Post Backup Event :D
hMailServer build LIVE on my servers: 5.4-B2014050402
#hmailserver on FreeNode IRC https://webchat.freenode.net/?channels=#hmailserver
*** ABSENT FROM hMail! Those in IRC know how to find me if urgent. ***

User avatar
mattg
Moderator
Moderator
Posts: 21257
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: I am alive

Post by mattg » 2011-07-29 03:20

Do the backups / restore work without being zipped?
I've had issues in earlier versions that the restore only worked from a zipped version.
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

Bill48105
Developer
Developer
Posts: 6192
Joined: 2010-04-24 23:16
Location: Michigan, USA

Re: I am alive

Post by Bill48105 » 2011-07-29 05:49

mattg wrote:Do the backups / restore work without being zipped?
I've had issues in earlier versions that the restore only worked from a zipped version.
What sort of problems matt? Maybe was just old hmail & has been fixed? I was curious & was doing backups then restores with 3GB of Data and worked fine but doubt I can say I could possibly test every possible scenario. :D Btw the only reason I was to do that much was due to the bug when using mssql ce but doing it FOUND that bug and also to me seemed so show maybe the limit should only be imposed if compression is checked (at least until 7z or such gets implemented).
Thx
Bill
hMailServer build LIVE on my servers: 5.4-B2014050402
#hmailserver on FreeNode IRC https://webchat.freenode.net/?channels=#hmailserver
*** ABSENT FROM hMail! Those in IRC know how to find me if urgent. ***

User avatar
mattg
Moderator
Moderator
Posts: 21257
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: I am alive

Post by mattg » 2011-07-29 07:37

I just recall a time when hmailserver built in restore worked fine, but only for compressed backups. It could never restore a non-compressed backup.

Been a while since I tested.
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

Bill48105
Developer
Developer
Posts: 6192
Joined: 2010-04-24 23:16
Location: Michigan, USA

Re: I am alive

Post by Bill48105 » 2011-07-29 15:03

mattg wrote:I just recall a time when hmailserver built in restore worked fine, but only for compressed backups. It could never restore a non-compressed backup.

Been a while since I tested.
Yeah I'll need to do some more testing but it appears the XML is always compressed and in either case the zip's need to follow specific naming. I'll do some more testing because a backup isn't much good if it can't be restore. :D
Thx
Bill
hMailServer build LIVE on my servers: 5.4-B2014050402
#hmailserver on FreeNode IRC https://webchat.freenode.net/?channels=#hmailserver
*** ABSENT FROM hMail! Those in IRC know how to find me if urgent. ***

rilex
Normal user
Normal user
Posts: 49
Joined: 2011-04-14 20:05

Re: I am alive

Post by rilex » 2011-07-31 21:15

Why not build a VSS Writer for hMailServer (ok, so it isn't that simple :))? That would solve the consistency issue, and you can just compress the VSS snapshot with what ever mechanism you decide to go with.

Any progress on resolving the COM crash issue? I'm holding off on working on my project to see this resolved in case some large changes in the COM API are made.

Bill48105
Developer
Developer
Posts: 6192
Joined: 2010-04-24 23:16
Location: Michigan, USA

Re: I am alive

Post by Bill48105 » 2011-07-31 21:48

rilex wrote:Why not build a VSS Writer for hMailServer (ok, so it isn't that simple :))? That would solve the consistency issue, and you can just compress the VSS snapshot with what ever mechanism you decide to go with.

Any progress on resolving the COM crash issue? I'm holding off on working on my project to see this resolved in case some large changes in the COM API are made.
Yeah VSS was discussed & perhaps something to dig into but the plan is to do as little as possible to 5.4 so it can get released finally.
As I said in my PM I'll look at the example but since the code needs to be run on the local server or network it doesn't appear to be super urgent but definitely something to look at.
Thx
Bill
hMailServer build LIVE on my servers: 5.4-B2014050402
#hmailserver on FreeNode IRC https://webchat.freenode.net/?channels=#hmailserver
*** ABSENT FROM hMail! Those in IRC know how to find me if urgent. ***

Kelden
Normal user
Normal user
Posts: 44
Joined: 2011-05-02 13:58

Re: I am alive

Post by Kelden » 2011-08-02 21:27

You can get a working VSS Backupscript here
http://code.google.com/p/hmailserver-ba ... loads/list

Bill48105
Developer
Developer
Posts: 6192
Joined: 2010-04-24 23:16
Location: Michigan, USA

Re: I am alive

Post by Bill48105 » 2011-08-02 22:59

Kelden wrote:You can get a working VSS Backupscript here
http://code.google.com/p/hmailserver-ba ... loads/list
Thanks for that but might be better posted up in the scripting section maybe instead of as a response to here..
In terms of your scripts it looks cool but 1 huge thing I see right off:

Code: Select all

::Only needed with integrated MSSQL / MYSQL Database
@echo Stopping hMailServer>%LOGFILE%
net stop hMailServer>>%LOGFILE%
If one must stop hmailserver what is point of doing shadow copy? :D

Or perhaps hmail is only stopped long enough to initiate the shadow copy then started & backup can commence with hmail running the rest of the time? If that is the case not ideal but not as huge of a deal but instead we might look at why stopping hmail is needed (perhaps a stop of services via COM command vs the service instead would suffice?) and maybe get a change made to hmail for this to work more smoothly.

Anyway, as I said probably best to post this up in user contributed scripts or scripting section & we can discuss it there.
Thx
Bill
hMailServer build LIVE on my servers: 5.4-B2014050402
#hmailserver on FreeNode IRC https://webchat.freenode.net/?channels=#hmailserver
*** ABSENT FROM hMail! Those in IRC know how to find me if urgent. ***

User avatar
mattg
Moderator
Moderator
Posts: 21257
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: I am alive

Post by mattg » 2011-08-03 04:37

Bill48105 wrote:If one must stop hmailserver what is point of doing shadow copy? :D
I've actually been experimenting with just stopping the hmailserver via the COM API rather than stopping the service

On my not very busy server it works well.

I guess the idea of stopping is so that you don't have mail coming in or going out for a few moments while the backup occurs.
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

Bill48105
Developer
Developer
Posts: 6192
Joined: 2010-04-24 23:16
Location: Michigan, USA

Re: I am alive

Post by Bill48105 » 2011-08-03 05:41

mattg wrote:I've actually been experimenting with just stopping the hmailserver via the COM API rather than stopping the service

On my not very busy server it works well.

I guess the idea of stopping is so that you don't have mail coming in or going out for a few moments while the backup occurs.
I haven't tried stopping via COM API but cool as I'd agree that seems like safer less obtrusive way to go about it rather than stopping the actual service assuming the need is to stop mail flow/access vs needing hmail to release the built-in database files.

The whole point of shadow copies is to be able to backup live in-use files but also done so in a snapshot at that moment (as in Windows takes note of changes from start to finish to provide the requester files as they were at time requested not as they are after changes) which in essence in a 2 birds 1 stone sort of way should eliminate need to stop hmail at all. That is the point or why use it?!? LOL I mean if you stop hmail no changes will occur & no files will be in use so just copy or back them up & not worry about changes. :D

Anyhow we're littering up Mr Martin's thread so hopefully Kelden starts up his own thread & we can carry on there as I've love to hear the answer. :)
Bill
hMailServer build LIVE on my servers: 5.4-B2014050402
#hmailserver on FreeNode IRC https://webchat.freenode.net/?channels=#hmailserver
*** ABSENT FROM hMail! Those in IRC know how to find me if urgent. ***

prisma
Senior user
Senior user
Posts: 310
Joined: 2010-07-09 13:16

Re: I am alive

Post by prisma » 2011-08-03 12:23

Bill48105 wrote:If one must stop hmailserver what is point of doing shadow copy?
There a many reasons to do VSCs. But fact seems to be, that hmailserver has got a problem if a snapshot is done with VSS (under specific circumstances?). We're using hmailserver within a VMWARE-VM running on VMWARE-Server 2.0. This virtualisation is using VSS to create snapshots. Every time I did a snapshot while hmailserver running, i had a dead hmailserver throwing OLE DB errors. After net stop/start everything is fine again.

So... creating a VSS-listener(writer?) for hmailserver itself and/or for it's backup would absolutely make sense. But not for a 5.4, I agree with you :)

Kelden
Normal user
Normal user
Posts: 44
Joined: 2011-05-02 13:58

Re: I am alive

Post by Kelden » 2011-08-04 19:46

The reason I'm stopping hMS service is that the MS SQL CE doesn't support VSS and won't be in
a consistent state without stopping the service. Only stopping hMS couldn't be enough because
hMS will still have the DB opened.

You don't need to stop hMS if you use real MS SQL Server.
I don't know about MySQL support VSS.

Kelden

rjk
Normal user
Normal user
Posts: 248
Joined: 2010-03-30 19:30
Location: uʍop ǝpısdn

Re: I am alive

Post by rjk » 2011-08-04 19:49

Bill48105 wrote:Thanks for that but might be better posted up in the scripting section maybe instead of as a response to here..
...
Anyway, as I said probably best to post this up in user contributed scripts or scripting section & we can discuss it there.
Thx
Bill
hMailServer 5.5.2-B2129 on Server 2008 R2 VM
MySQL 5.5.25, IIS 7.5, PHP 5.6.2 via FastCGI, RoundCube 1.0.3
XenServer 6.0 on HP DL380 G5 32GB RAM

User avatar
Spyd
Normal user
Normal user
Posts: 43
Joined: 2008-02-19 10:15
Location: Barcelona, Spain
Contact:

Re: I am alive

Post by Spyd » 2011-10-07 11:54

I'm just curious about what happened to Martin because his last activity in this forum is more than 2 months old.
Does anybody know?

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: I am alive

Post by ^DooM^ » 2011-10-07 12:08

He was last active on the forum on 25th August which is 6 weeks and not "more than two months". Aside that though your guess is as good as mine. Why do you need him?
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

Post Reply