Store partial path to email files in database

This forum contains features that has been archived. This section contains implemented features, duplicate requests, and requests which we have decided not to implement.
Post Reply

Usefull?

Yes
29
100%
No
0
No votes
 
Total votes: 29

mafr1
Normal user
Normal user
Posts: 66
Joined: 2009-04-01 18:39
Location: NL

Store partial path to email files in database

Post by mafr1 » 2009-09-01 19:44

Storing only the partial path to an email file in the database, would make changing the DataFolder location easier.

This way a path change in the .ini file and a restart from the hMailServer service will change the DataFolder location. This will eliminate the need for database updates.
Marco
hMailServer 5.3.1-B1748

User avatar
sheffters
Senior user
Senior user
Posts: 453
Joined: 2009-07-01 20:46
Contact:

Re: Store partial path to email files in database

Post by sheffters » 2009-09-01 19:53

handy nice to have.

S.

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: Store partial path to email files in database

Post by ^DooM^ » 2009-09-01 21:07

I could have sworn there was already a feature request for this because I remember discussing this about a year ago. I can't seem to find it though. Added my vote :)
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: Store partial path to email files in database

Post by ^DooM^ » 2009-09-01 21:14

Topic discussed here if anyones interested.

http://www.hmailserver.com/forum/viewto ... 12&p=83700
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

User avatar
martin
Developer
Developer
Posts: 6834
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: Store partial path to email files in database

Post by martin » 2010-01-06 23:11

I've implemented this for hMailServer 5.4. It's not really perfect, but probably as good as it gets unless someone has a good idea on how to fix the remaining oddities.

Paths for old messages won't be automatically removed though. Server administrators have to run Data Directory Synchronizer to update the paths for existing messages. The reason is that updating the existing paths is a time consuming process. Messages may be stored in different locations, in different folder levels (domain\account\message.eml vs domain\account\me\message.eml). If this would be done during the upgrade process, this would add quite a lot of downtime to the upgrade process.

The data directory synchronizer in 5.4 will go through the entire data directory. If a message is found in a place which doesn't conform to the naming standard for files in the data directory, the message file will be moved to the correct location. For instance, if a message is located in C:\DataDir\example.com\account\message.eml, it will be moved to C:\DataDir\example.com\account\{0C}\{0CD3719B-34CA-49D6-BC41-A8405191994D}.eml)

I have thought about updating the hm_messages table automatically after the system has been upgraded to 5.4. But this will be a quite resource intensive operation and I feel that the system administrator need to be able to control when its run. This is why its a part of data directory synchronizer.

For new messages, only the file name ({0CD3719B-34CA-49D6-BC41-A8405191994D}.eml in this case) is stored in the database. hMailServer will assume that the file is located under <DataDirectory><Domain><Account><FirstTwoCharactersOfFileName><FileName>

In the diagnostics section in hMailServer Administrator, a new test which confirms two things have been added:
  • That only the file name is stored in the hm_messages table (rather than the full path).
  • That all files are located under the data directory and not split up into different locations
So in 5.4, most users won't have to tweak the database if they want to move the data directory. The reason where this will still be needed is if the user has split up the data directory to two different locations. For instance, some files may have been stored on C:\ while other files have been stored on D:\. In this case, the user needs to manually move the message files and update the database.

I'll put together a tutorial on how to move the data directory later.

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: Store partial path to email files in database

Post by ^DooM^ » 2010-01-07 02:04

sounds like the best option to me. Nice work as usual!
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

User avatar
mattg
Moderator
Moderator
Posts: 20293
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Store partial path to email files in database

Post by mattg » 2010-01-07 02:14

Will the data directory synchroniser still put 'found' emails into the INBOX?
Does this mean that someone who wants to update the paths for their entire database will need to re-file all mails into other IMAP folders?
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
martin
Developer
Developer
Posts: 6834
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: Store partial path to email files in database

Post by martin » 2010-01-07 20:57

mattg,
DDS only puts the messages into the Inbox if the database contains no information on what IMAP folder the messages are located in. (If the message file is not listed in the database). If you run DDS on your system you should see that the messages remains in the IMAP folder they are already in.

So no, people won't have to re-file all mails into the different IMAP folders; hMailServer will just update the paths and possibly move the .EML files in the data directory around a bit if they aren't in the right locations.

User avatar
mattg
Moderator
Moderator
Posts: 20293
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Store partial path to email files in database

Post by mattg » 2010-01-08 03:01

Thanks Martin,
I've not used the DDS much, I was just confirming that you had thought about the impact of 'results into inbox' that we hear occasionally on the forum.

I was thinking that perhaps if a message didn't match the path stored in the database before the DDS started that it might become an 'unknown message'.

All good, I should know better than to check. You are always on the ball. :D

Matt
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

sckramer2
Normal user
Normal user
Posts: 133
Joined: 2009-07-31 21:50

Re: Store partial path to email files in database

Post by sckramer2 » 2010-01-12 18:21

Looks like there will be no bloat in the database from not storing all those redundant paths... cool

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: Store partial path to email files in database

Post by ^DooM^ » 2010-04-07 18:00

This is implemented in 5.4, moved to archive
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

Post Reply