I am alive

Post by **martin** » 2011-07-25 00:49

I know it was quite some time since I was active around here. I also know some of you are waiting for a stable version 5.4. There are a few things (mainly small stuff) which needs to be fixed before we can release 5.4 as stable. I will start to work on that now.

For instance, one thing which is remaining is that the backup/restore functionality does not work properly now when we no longer store the full paths in the database.

Martin

Bill48105 · Post by **Bill48105** » 2011-07-25 01:44

Hey Martin,
Welcome back! Yeah the inability to open messages in queue should be fixed too. (Work-around is to modify the shortcut to 'Start in' the Data folder but I'd imagine the Data folder should be pre-pended to the file path in the GUI)

As far as backup/restore, wow it'd be great to fix it so it works on bigger data sets as the ~1GB limit isn't ideal but I haven't even looked at the code to see if there would be an easy fix or if it needs to wait til 5.4.1 to avoid delaying 5.4 for now. Maybe I'll dig into that tonight as now I'm curious & had been meaning to look but keep getting sidetracked.

Yell if there are any of those things you want me to work on so we're not duplicating efforts.

But WOO WOO! is what I say!

Bill

mattg · Post by **mattg** » 2011-07-25 02:59

Welcome back Martin.

Hope all is good in your life etc...

Matt

bescher · Post by **bescher** » 2011-07-25 10:15

Welcome Back Martin Great to see you. I agree with Bill on the backup/restore if it can be larger 2 gig or even 2.5 would be fantastic.

Now all we need is Nico to pop up with a newer version of his Clamav and the Clamav extra databases

Post by **martin** » 2011-07-25 13:09

The problem with the 2GB backup limitation is that hMailServer uses a ZIP library incapable of larger files than 2GB. I think that the backup/restore functionality needs to be revamped a bit. It's slow and it's not transactional which may cause confusion. For instance, when restoring a backup a 'ghost' message may appear (same message in two folders), even if it's unlikely.

Bill48105 · Post by **Bill48105** » 2011-07-25 14:54

Think 7z lib is avail but need to confirm compatible with hmail's licensing. Didn't want to sidetrack getting 5.4 released this could very well wait until 5.4.1 if needed, just mentioning something that seems to come up quite a bit that would make hmail immensely more user-friendly not only for backing up but for migrating between servers etc.

As I said let me know which parts if any you want me to work on to get this thing moved forward.
Thx!
Bill

katip · Post by **katip** » 2011-07-25 19:04

bescher wrote:Welcome Back Martin Great to see you. I agree with Bill on the backup/restore if it can be larger 2 gig or even 2.5 would be fantastic.

Now all we need is Nico to pop up with a newer version of his Clamav and the Clamav extra databases

+1 (Nicooo!)

Hi Martin, good to see you again.

Bill48105 · Post by **Bill48105** » 2011-07-25 19:15

Am I missing something? Do people think martin has special "Bat phone" link to Nico or something? LOL

Looking at the GUI code it looks like there is no (obvious) reason why there is a 1.5G limit on backup as long as the user does not check 'compress' because all that happens in that case is a DIR copy is made. Should be possible to do external compressor or even 7z though.

Again IMO top priority should be to get final touches done to get 5.4 out finally so don't want to distract from that. The comment about backup/restore had just reminded me of the hassles of the the 1.5G limit for many people. Obviously the path stuff needs to be fixed either way.
Bill

rjk · Post by **rjk** » 2011-07-25 19:41

Good to hear from you, Martin!
I think 7z would be a useful addition to 5.4.1

^DooM^ · Post by **^DooM^** » 2011-07-25 19:45

7Zip has my vote too.

mattg · Post by **mattg** » 2011-07-26 08:12

^DooM^ wrote:7Zip has my vote too.

Yep, me too

bescher · Post by **bescher** » 2011-07-26 11:25

No Bill I do not think Martin has a direct link to Nico. I was just mentioning another part of my dream list

prisma · Post by **prisma** » 2011-07-28 09:58

mattg wrote:
^DooM^ wrote:7Zip has my vote too.
Yep, me too

Yep, me too. An improvement of the backup seems to be necessary and would be good reason to wait some more weeks. I like the build in backup very much, but we have nearly reached the 2gig limitation. So, I'd be the first who likes to test it !! A good reason to update

Welcome back Martin!

Bill48105 · Post by **Bill48105** » 2011-07-28 15:46

prisma wrote:Yep, me too. An improvement of the backup seems to be necessary and would be good reason to wait some more weeks. I like the build in backup very much, but we have nearly reached the 2gig limitation. So, I'd be the first who likes to test it !! A good reason to update

Welcome back Martin!

FYI: I have tested & found a bug in the backup system that does not stop it at 1.5G for mssql ce as expected. In doing so it appears I have confirmed what I believed was true: If user does not choose compress backup then there really is no real reason to restrict to 1.5G. Am thinking of making the 1.5G limit only enforced if the user chooses compress for now while fixing the bug. That would allow people who have > 1.5G to still use internal backup as long as they don't compress (zip) it. Then they can compress the backup folder using whatever tool they choose after backup completes. That'll let us not hold up 5.4 & maybe try to get 7z in for 5.4.1.

Slug · Post by **Slug** » 2011-07-28 15:50

bescher wrote:No Bill I do not think Martin has a direct link to Nico. I was just mentioning another part of my dream list

I have emailed Nico a few times now with zero response, not sure if he is still alive

I hope so.

And 7zip has my vote as well ..

Michael

Post by **martin** » 2011-07-28 17:49

I've fixed the problem with viewing messages in Administrator. Currently creating a new build.

Regarding backup compression. A couple of months ago, I spent some time trying to find an open source alternative which I could use in hMailServer. 7zip was one of the alternatives I looked into. I can't recall exactly what the problem was, but while it was open source, I came to the conclusion that i had to use some part of 7zip which was not compatible with the hMailServer licensing. But when I look into it now, the 7zip SDK seems to be public domain which means that it shouldn't be a problem. To make it really simple, hMailServer could just launch the 7zip executable and perform the compression, rather than trying to integrate the 7zip engine into hMailServer. I'll contact the 7zip author and check.

^DooM^ · Post by **^DooM^** » 2011-07-28 20:08

If that doesn't pan out martin just allow us to enter a string to be ran after backup so we can use whatever compression program we want in much the same way the AV works.

Bill48105 · Post by **Bill48105** » 2011-07-28 22:54

^DooM^ wrote:If that doesn't pan out martin just allow us to enter a string to be ran after backup so we can use whatever compression program we want in much the same way the AV works.

Or Post Backup Event

mattg · Post by **mattg** » 2011-07-29 03:20

Do the backups / restore work without being zipped?
I've had issues in earlier versions that the restore only worked from a zipped version.

Bill48105 · Post by **Bill48105** » 2011-07-29 05:49

mattg wrote:Do the backups / restore work without being zipped?
I've had issues in earlier versions that the restore only worked from a zipped version.

What sort of problems matt? Maybe was just old hmail & has been fixed? I was curious & was doing backups then restores with 3GB of Data and worked fine but doubt I can say I could possibly test every possible scenario.

Btw the only reason I was to do that much was due to the bug when using mssql ce but doing it FOUND that bug and also to me seemed so show maybe the limit should only be imposed if compression is checked (at least until 7z or such gets implemented).
Thx
Bill

mattg · Post by **mattg** » 2011-07-29 07:37

I just recall a time when hmailserver built in restore worked fine, but only for compressed backups. It could never restore a non-compressed backup.

Been a while since I tested.

Bill48105 · Post by **Bill48105** » 2011-07-29 15:03

mattg wrote:I just recall a time when hmailserver built in restore worked fine, but only for compressed backups. It could never restore a non-compressed backup.

Been a while since I tested.

Yeah I'll need to do some more testing but it appears the XML is always compressed and in either case the zip's need to follow specific naming. I'll do some more testing because a backup isn't much good if it can't be restore.

Thx
Bill

rilex · Post by **rilex** » 2011-07-31 21:15

Why not build a VSS Writer for hMailServer (ok, so it isn't that simple

)? That would solve the consistency issue, and you can just compress the VSS snapshot with what ever mechanism you decide to go with.

Any progress on resolving the COM crash issue? I'm holding off on working on my project to see this resolved in case some large changes in the COM API are made.

Bill48105 · Post by **Bill48105** » 2011-07-31 21:48

rilex wrote:Why not build a VSS Writer for hMailServer (ok, so it isn't that simple )? That would solve the consistency issue, and you can just compress the VSS snapshot with what ever mechanism you decide to go with.

Any progress on resolving the COM crash issue? I'm holding off on working on my project to see this resolved in case some large changes in the COM API are made.

Yeah VSS was discussed & perhaps something to dig into but the plan is to do as little as possible to 5.4 so it can get released finally.
As I said in my PM I'll look at the example but since the code needs to be run on the local server or network it doesn't appear to be super urgent but definitely something to look at.
Thx
Bill

Kelden · Post by **Kelden** » 2011-08-02 21:27

You can get a working VSS Backupscript here
http://code.google.com/p/hmailserver-ba ... loads/list

Bill48105 · Post by **Bill48105** » 2011-08-02 22:59

Kelden wrote:You can get a working VSS Backupscript here
http://code.google.com/p/hmailserver-ba ... loads/list

Thanks for that but might be better posted up in the scripting section maybe instead of as a response to here..
In terms of your scripts it looks cool but 1 huge thing I see right off:

Code: Select all

::Only needed with integrated MSSQL / MYSQL Database
@echo Stopping hMailServer>%LOGFILE%
net stop hMailServer>>%LOGFILE%

If one must stop hmailserver what is point of doing shadow copy?

Or perhaps hmail is only stopped long enough to initiate the shadow copy then started & backup can commence with hmail running the rest of the time? If that is the case not ideal but not as huge of a deal but instead we might look at why stopping hmail is needed (perhaps a stop of services via COM command vs the service instead would suffice?) and maybe get a change made to hmail for this to work more smoothly.

Anyway, as I said probably best to post this up in user contributed scripts or scripting section & we can discuss it there.
Thx
Bill

mattg · Post by **mattg** » 2011-08-03 04:37

Bill48105 wrote:If one must stop hmailserver what is point of doing shadow copy?

I've actually been experimenting with just stopping the hmailserver via the COM API rather than stopping the service

On my not very busy server it works well.

I guess the idea of stopping is so that you don't have mail coming in or going out for a few moments while the backup occurs.

Bill48105 · Post by **Bill48105** » 2011-08-03 05:41

mattg wrote:I've actually been experimenting with just stopping the hmailserver via the COM API rather than stopping the service

On my not very busy server it works well.

I guess the idea of stopping is so that you don't have mail coming in or going out for a few moments while the backup occurs.

I haven't tried stopping via COM API but cool as I'd agree that seems like safer less obtrusive way to go about it rather than stopping the actual service assuming the need is to stop mail flow/access vs needing hmail to release the built-in database files.

The whole point of shadow copies is to be able to backup live in-use files but also done so in a snapshot at that moment (as in Windows takes note of changes from start to finish to provide the requester files as they were at time requested not as they are after changes) which in essence in a 2 birds 1 stone sort of way should eliminate need to stop hmail at all. That is the point or why use it?!? LOL I mean if you stop hmail no changes will occur & no files will be in use so just copy or back them up & not worry about changes.

Anyhow we're littering up Mr Martin's thread so hopefully Kelden starts up his own thread & we can carry on there as I've love to hear the answer.

Bill

prisma · Post by **prisma** » 2011-08-03 12:23

Bill48105 wrote:If one must stop hmailserver what is point of doing shadow copy?

There a many reasons to do VSCs. But fact seems to be, that hmailserver has got a problem if a snapshot is done with VSS (under specific circumstances?). We're using hmailserver within a VMWARE-VM running on VMWARE-Server 2.0. This virtualisation is using VSS to create snapshots. Every time I did a snapshot while hmailserver running, i had a dead hmailserver throwing OLE DB errors. After net stop/start everything is fine again.

So... creating a VSS-listener(writer?) for hmailserver itself and/or for it's backup would absolutely make sense. But not for a 5.4, I agree with you

Kelden · Post by **Kelden** » 2011-08-04 19:46

The reason I'm stopping hMS service is that the MS SQL CE doesn't support VSS and won't be in
a consistent state without stopping the service. Only stopping hMS couldn't be enough because
hMS will still have the DB opened.

You don't need to stop hMS if you use real MS SQL Server.
I don't know about MySQL support VSS.

Kelden

rjk · Post by **rjk** » 2011-08-04 19:49

Bill48105 wrote:Thanks for that but might be better posted up in the scripting section maybe instead of as a response to here..
...
Anyway, as I said probably best to post this up in user contributed scripts or scripting section & we can discuss it there.
Thx
Bill

Spyd · Post by **Spyd** » 2011-10-07 11:54

I'm just curious about what happened to Martin because his last activity in this forum is more than 2 months old.
Does anybody know?

^DooM^ · Post by **^DooM^** » 2011-10-07 12:08

He was last active on the forum on 25th August which is 6 weeks and not "more than two months". Aside that though your guess is as good as mine. Why do you need him?

I am alive

I am alive

Re: I am alive

Re: I am alive

Re: I am alive

Re: I am alive

Re: I am alive

Re: I am alive

Re: I am alive

Re: I am alive

Re: I am alive

Re: I am alive

Re: I am alive

Re: I am alive

Re: I am alive

Re: I am alive

Re: I am alive

Re: I am alive

Re: I am alive

Re: I am alive

Re: I am alive

Re: I am alive

Re: I am alive

Re: I am alive

Re: I am alive

Re: I am alive

Re: I am alive

Re: I am alive

Re: I am alive

Re: I am alive

Re: I am alive

Re: I am alive

Re: I am alive

Re: I am alive