hMail broke

Forum for things that doesn't really have anything to do with hMailServer. Such as php.ini, beer, etc etc.
Post Reply
^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

hMail broke

Post by ^DooM^ » 2008-05-17 15:20

Well my backup mail server broke. This is the first time in all my years using hMail that I have had a problem that wasn't my own fault :P The greylisting triplets table became corrupted somehow. A quick repair on the table soon fixed it though. Now the difference between my main mail server and my backup server is that the backup server is using the built in database. I have noticed over the years that most other corrupted tables were also using the built in database. I have never encountered a problem using an external database. This begs the question, why would the built in database be more susceptible to corrupted tables than an external standalone version?

Just wondering.
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

Kaan1983
Senior user
Senior user
Posts: 595
Joined: 2007-01-30 16:26
Location: TÜRKIYE

Re: hMail broke

Post by Kaan1983 » 2008-05-17 16:13

Well that subject is way over my head but nice to hear that you experienced an error :P

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: hMail broke

Post by ^DooM^ » 2008-05-17 17:17

Kaan1983 wrote:Well that subject is way over my head but nice to hear that you experienced an error :P
lol well never nice to have any errors but to be the first semi serious one I have had that was not of my own doing definitely proves hMails stability to me. The error was easily fixed, just unencrypted the password using the tool provided, logged in using the command line server from a command prompt

Code: Select all

mysql -u root -P 3307 -p
entered the unencrypted password selected the hmail database

Code: Select all

use hmailserver
and repaired the table

Code: Select all

repair table hm_greylisting_triplets
and voila, all working again.

As this was my backup server I have noticed it is getting slammed by spam bots as of late even with all the precautions I have in place to deter them. I have to that end altered my greylisting options as follows. Defer mail for 20 minutes, remove unused records after 1 day, remove used records after 2 days. Hopefully this will keep my GY table small and force spambots to hit my main mailserver rather than abuse my baby backup server :)
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

Kaan1983
Senior user
Senior user
Posts: 595
Joined: 2007-01-30 16:26
Location: TÜRKIYE

Re: hMail broke

Post by Kaan1983 » 2008-05-17 17:36

^DooM^ wrote:lol well never nice to have any errors but to be the first semi serious one I have had that was not of my own doing definitely proves hMails stability to me.
Well as an engineer I'd be jobless if there wouldn't be any errors and problems :) So, in a way, I enjoy em :wink:
^DooM^ wrote:Hopefully this will keep my GY table small and force spambots to hit my main mailserver rather than abuse my baby backup server :)
:D
I wish you SPAM free days...

Carl
Normal user
Normal user
Posts: 111
Joined: 2008-03-18 17:33
Location: Utah, USA
Contact:

Re: hMail broke

Post by Carl » 2008-05-18 02:32

Not a developer, programmer, or even certified IT guy. (Certified crazy, but that's another story. :P) But at a guess it seems like going to an external database would require one more (very small) level of communications. Could be wrong, ignorant, or both though. That's the only possible difference I can think of.
Carl

98% of computer users know just enough to be dangerous, but not enough to be useful. I'm afraid I don't fall in the top 2%.

Kaan1983
Senior user
Senior user
Posts: 595
Joined: 2007-01-30 16:26
Location: TÜRKIYE

Re: hMail broke

Post by Kaan1983 » 2008-05-18 09:35

Tell us about the 'Certified Crazy' story :D

User avatar
Slug
Moderator
Moderator
Posts: 1369
Joined: 2005-03-13 05:42
Location: Sydney Australia
Contact:

Re: hMail broke

Post by Slug » 2008-05-19 16:11

Or perhaps the config for the internal V external DB is different.

I am the reverse only my back up mail server uses a external DB and I remember something about InnoDB or MyISAM DB setup ..

Just a guess ....

Michael
Missing Hmailserver ... Now running Debian servers

User avatar
maggiore81
Normal user
Normal user
Posts: 171
Joined: 2008-01-11 16:02
Location: near Ravenna (Italy)
Contact:

Re: hMail broke

Post by maggiore81 » 2008-05-26 14:16

Hello Doom.

Are you Doom 1, 2, or 3 ? :)

Well.. I just read this topic and it comes to my mind this:

If I use the internal DB, how can I plan manteinance over the DB?
For example if I use SQL server, I could do maintenace on it, maintenance plan, etc... but what over internal MYSQL?
I have a mailserver running with the internal DB (for ease of use) and I am feared to what could happen if something in the DB gets corrupted. I do daily backups of the HMAIL folder, so in case of disaster I could restore it entirely... but what about the DB, any maintenance plan to be scheduled?

Thank you
Mr. Spadoni
Network Administrator
--
Spadhausen ISP
admin (at) spadhausen . com

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: hMail broke

Post by ^DooM^ » 2008-05-26 14:53

Internal MySQL is exactly the same as the External one just its local to hMail and runs on port 3307. Connect to it like you would any other SQL server and run your maintenance on it.

Follow these instructions on how to gain access to the MySQL Server.

http://www.hmailserver.com/documentatio ... pair_mysql
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

User avatar
maggiore81
Normal user
Normal user
Posts: 171
Joined: 2008-01-11 16:02
Location: near Ravenna (Italy)
Contact:

Re: hMail broke

Post by maggiore81 » 2008-05-26 15:04

Hello, thank you for your quick answer.
I have already gained access to the MYSQL database when tried to install RoundCube.

I have now access to the DB. I can use any maintenance plan for any MYSQL DB?

To access the DB I Use the SQLyog utility.
Mr. Spadoni
Network Administrator
--
Spadhausen ISP
admin (at) spadhausen . com

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: hMail broke

Post by ^DooM^ » 2008-05-26 15:08

maggiore81 wrote:I can use any maintenance plan for any MYSQL DB?
Just run an optimize on all tables and a repair as well if you want to, it won't hurt to run a repair on a non broken table.
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

westdam
Senior user
Senior user
Posts: 728
Joined: 2006-08-01 21:24
Location: Padova, Italy
Contact:

Re: hMail broke

Post by westdam » 2008-05-26 16:57

well, i've notice this about 1 years ago..then i've moved from internal to external mysql and then to MSSQL2005 express
.With MSSQL i dont have any clue.. dont know how... it's strange..

User avatar
maggiore81
Normal user
Normal user
Posts: 171
Joined: 2008-01-11 16:02
Location: near Ravenna (Italy)
Contact:

Re: hMail broke

Post by maggiore81 » 2008-05-26 17:00

Thanks for your complete answer.

Well.. I could plan to migrate to SQL Express 2005 SP2, but I have noticed (and I did not find any docs about that) that SQL 2005 (2000 works well) cannot install on Via C3 CPU.
I have a mailserver running on a Via C3 900MHz with 768Mb SDRAM (2x80gb HDD EIDE UDMA100 RAID) and works perfectly with SQL 2000 SP4 and MYSQL, but NOT on SQL2005 (no compatible CPU found).
Mr. Spadoni
Network Administrator
--
Spadhausen ISP
admin (at) spadhausen . com

westdam
Senior user
Senior user
Posts: 728
Joined: 2006-08-01 21:24
Location: Padova, Italy
Contact:

Re: hMail broke

Post by westdam » 2008-05-26 17:36

mmmm
strange...
never tried on via C3 cpu but i dont believe a c3 cpu can handle much mails...

User avatar
maggiore81
Normal user
Normal user
Posts: 171
Joined: 2008-01-11 16:02
Location: near Ravenna (Italy)
Contact:

Re: hMail broke

Post by maggiore81 » 2008-05-26 17:38

I put that test-server with the specs I wrote, and worked very fast.

The problem with SQL 2005 is that SQL engine is not supported on via CPU.
Mr. Spadoni
Network Administrator
--
Spadhausen ISP
admin (at) spadhausen . com

bmorejon
Normal user
Normal user
Posts: 43
Joined: 2007-06-01 18:48

Re: hMail broke

Post by bmorejon » 2008-07-30 21:31

My table hm_messages just got corrupted. :(

bmorejon
Normal user
Normal user
Posts: 43
Joined: 2007-06-01 18:48

Re: hMail broke

Post by bmorejon » 2008-07-31 23:45

What does the Repair command do to repair the corrupt table ?

Noran Rad
New user
New user
Posts: 10
Joined: 2008-08-10 13:12

Re: hMail broke

Post by Noran Rad » 2008-08-10 18:41

Out of sheer curiosity, is the version of MySQL used by Hmail the same version you used as an external database?

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: hMail broke

Post by ^DooM^ » 2008-08-10 19:21

They are both V4, the build's are not the same though.
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

Noran Rad
New user
New user
Posts: 10
Joined: 2008-08-10 13:12

Re: hMail broke

Post by Noran Rad » 2008-08-10 19:28

Hmmm, well I guess that could be one piece to the puzzle, but then it may not mean anything at all...

I figured, worst case scenario it would be some interrupted API request.

Weirdest case scenario it could have been intentional on some spoofer's part. :|

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: hMail broke

Post by ^DooM^ » 2008-08-10 20:20

not sure how a spoofer could corrupt the greylist table, what happened was the auto increment field was out by one, it was trying to add a line to an existing row instead of a new one.
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

Noran Rad
New user
New user
Posts: 10
Joined: 2008-08-10 13:12

Re: hMail broke

Post by Noran Rad » 2008-08-11 08:04

Well, it was kind of a weird joke...

Don't mind me, sometimes I'm better off ignored.

roi
Normal user
Normal user
Posts: 153
Joined: 2009-09-20 12:56
Location: Chiba, Japan

Re: hMail broke

Post by roi » 2009-09-23 13:23

DooM... I was reading this old post of yours with much interest as I try to understand what to expect in the future. Your comment that internal DB has more chances of breaking than external DB was of especial interest since I am running an internal DB in my first baby steps running hMS.

It's everybody's guess why this might be so, but nobody can dispute your long experience with hMS. I would dare guess the bandwidth difference between internal and external makes the difference. With external DB, the DB engine has all the time to read/write because the network bandwidth is relatively slow. With internal DB, the DB engine can get overwhelmed with the very fast flow of commands and thus delayed in reading/writing tables which can get out of sync and corrupted.

On the surface, the only difference between internal and external DB would be the bandwidth and the speed by which the commands come. This is a guess.

I do thank you for your valuable observation... I should start thinking of porting to an external database.

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: hMail broke

Post by ^DooM^ » 2009-09-23 14:03

As a note of interest if you have installed V5 of hMail straight off the bat (meaning not upgraded from V4) and are using the built in database you should be fine. I haven't seen any reports of corrupted databases since Martin switched from MySQL to MSSQL.
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

roi
Normal user
Normal user
Posts: 153
Joined: 2009-09-20 12:56
Location: Chiba, Japan

Re: hMail broke

Post by roi » 2009-09-23 15:29

I read somewhere in the forum that MySQL could handle a larger database than the built-in MSSQL Express edition, so I opted for MySQL in my first install (V4.4 then V5.2).

I also understand record locking (as opposed to table locking) is relatively new to MySQL. Don't know if this has/had something to do with MySQL instability (don't know if I am talking of past or present) when the DB is internal.

From my very limited knowledge, it seems the MySQL-MSSQL choice is sort of "gain one, lose one". Oh well... I'll still learn from your experience, and keep in mind the internal/external DB issue.

Either way, I don't see too many posts in the forum on "non-human" caused corruption of DB and that speaks very well of hMS in general. As is true with most DB, manually tweak part of the data and you risk breaking the whole DB so most of the posts on corrupt DB were self-inflicted.

Your example is very rare in the forum and a very valuable piece of information.

User avatar
mattg
Moderator
Moderator
Posts: 20799
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: hMail broke

Post by mattg » 2009-09-23 15:37

roi wrote:I would dare guess the bandwidth difference between internal and external makes the difference. With external DB, the DB engine has all the time to read/write because the network bandwidth is relatively slow. With internal DB, the DB engine can get overwhelmed with the very fast flow of commands and thus delayed in reading/writing tables which can get out of sync and corrupted.
How exactly do you suppose that the 'internal' MySQL in version 4 was different than an 'external' version?
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

roi
Normal user
Normal user
Posts: 153
Joined: 2009-09-20 12:56
Location: Chiba, Japan

Re: hMail broke

Post by roi » 2009-09-23 16:08

I'm not assuming any particular version or version differences of the DB engine. Only that given a DB engine, that DB engine can be overwhelmed with a series of pending commands sent by the application if the bandwidth (internal bus) is super fast compared to the bandwidth (network) that is slow (even if 1Gb/sec network).

The succeeding DB table read/write and subsequent table/record locking mechanism can get out of sync with fast bandwidth and heavy application traffic. On the other hand slow network bandwidth can actually act as a traffic speed control to slow the arrival of app commands to the DB engine.

I am guessing, of course, but there should be no difference how a DB functions between internal and external DB. The big difference, if any, would be the bandwidth by which the app communicates with the DB engine. Logically, external DB would be more subject to corruption due to network data packet corruption, but with DooM's long experience and example, the opposite seems to be true -- internal DB is more susceptable to corruption than external DB. So I am guessing the slow bandwidth makes external DB more stable.

User avatar
mattg
Moderator
Moderator
Posts: 20799
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: hMail broke

Post by mattg » 2009-09-23 16:29

When we talk here of 'internal' vs 'external' database, we aren't necessarily meaning that an external database is running elsewhere on the LAN.

hMailserver ver 4 included a copy of the community version of MySQL, and was accessed on a non-standard port number. Many people who used an 'external' database would choose an existing installation of what was likely to be an identical version of MySQL, just that you used the standard ports. This was used so that you didn't unnecessarily have two versions of MySQL running. You could use an external machine elsewhere on the LAN, but that is not necessarily the case.
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

roi
Normal user
Normal user
Posts: 153
Joined: 2009-09-20 12:56
Location: Chiba, Japan

Re: hMail broke

Post by roi » 2009-09-23 16:55

Sorry... my misunderstanding of the meaning of external is obviously wrong. "External" means "not integrated" within hMS, right?. When I read DooM's post, I understood he meant DB external of the host running hMS via the network. If my understanding of external in the context of networked DB is wrong, then all my posts are totally in error... in which case, I apologize for my ignorance.
hMS: 5.2.1-B361 | DB: Internal MySQL from hMS 4.4 | OS: W2K3 1Gb VM

User avatar
mattg
Moderator
Moderator
Posts: 20799
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: hMail broke

Post by mattg » 2009-09-23 17:02

That's OK.

I recall that Dooms 'external' setup is on a separate machine, and so in his case you may be right.

Good set of thoughts though.
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: hMail broke

Post by ^DooM^ » 2009-09-23 18:24

In my case external meant another server on my LAN however for the most part when we say external database we mean not bundled with hMail.
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

User avatar
sheffters
Senior user
Senior user
Posts: 453
Joined: 2009-07-01 20:46
Contact:

Re: hMail broke

Post by sheffters » 2009-09-23 19:00

http://www.mysql.com/oem/

dont suppose it uses the MySQL Embedded Server?

Probably not as widely used / tested as much?

S.

^DooM^
Site Admin
Posts: 13861
Joined: 2005-07-29 16:18
Location: UK

Re: hMail broke

Post by ^DooM^ » 2009-09-23 19:24

Not entirely sure what version came with hMail for V4
If at first you don't succeed, bomb disposal probably isn't for you! ヅ

User avatar
Slug
Moderator
Moderator
Posts: 1369
Joined: 2005-03-13 05:42
Location: Sydney Australia
Contact:

Re: hMail broke

Post by Slug » 2009-09-26 16:50

^DooM^ wrote:Not entirely sure what version came with hMail for V4

I think it was 4.17.xxx
Missing Hmailserver ... Now running Debian servers

Post Reply