Multiple Instances on one DB Server

Use this forum if you have installed hMailServer and want to ask a question related to a production release of hMailServer. Before posting, please read the troubleshooting guide. A large part of all reported issues are already described in detail here.
Post Reply
reesebruen
New user
New user
Posts: 1
Joined: 2021-01-18 13:16

Multiple Instances on one DB Server

Post by reesebruen » 2021-01-18 13:26

Hi there,

I have a question where I couldn't find an answer yet.

Currently I'm running an instance of hMailserver on the embedded CE database. Now as I get quite a bit of emails a day, I would like
to move to a MySQL DB (at amazon RDS) where I expect a better performance.
For a single instance this works well.

Now my thoughts are, is it possible to have multiple hMailserver instances running on different servers, where both connect to the same IP?
Does anyone has experience on this? Or will it mess up the installation if you have multiple instances connecting to the same DB server?

Thanks for your help!

Cheers

palinka
Senior user
Senior user
Posts: 2450
Joined: 2017-09-12 17:57

Re: Multiple Instances on one DB Server

Post by palinka » 2021-01-18 15:49

reesebruen wrote:
2021-01-18 13:26
Hi there,

I have a question where I couldn't find an answer yet.

Currently I'm running an instance of hMailserver on the embedded CE database. Now as I get quite a bit of emails a day, I would like
to move to a MySQL DB (at amazon RDS) where I expect a better performance.
For a single instance this works well.

Now my thoughts are, is it possible to have multiple hMailserver instances running on different servers, where both connect to the same IP?
Does anyone has experience on this? Or will it mess up the installation if you have multiple instances connecting to the same DB server?

Thanks for your help!

Cheers
Short answer is no. Both hmailserver instances would need the same data folder. I doubt that's even possible, much less advisable.

If you explain your use case, perhaps we can help you setup your system to achieve what you want to do.

User avatar
johang
Senior user
Senior user
Posts: 441
Joined: 2008-09-01 09:20

Re: Multiple Instances on one DB Server

Post by johang » 2021-01-18 17:04

reesebruen wrote:
2021-01-18 13:26
Hi there,

I have a question where I couldn't find an answer yet.

Currently I'm running an instance of hMailserver on the embedded CE database. Now as I get quite a bit of emails a day, I would like
to move to a MySQL DB (at amazon RDS) where I expect a better performance.
For a single instance this works well.

Now my thoughts are, is it possible to have multiple hMailserver instances running on different servers, where both connect to the same IP?
Does anyone has experience on this? Or will it mess up the installation if you have multiple instances connecting to the same DB server?

Thanks for your help!

Cheers
its sorta hidden in Palinkas answer.. but let me be very clear

the emails are not stored in the database !!! ( yes you read it right ... no emails inside database )

hMailServer e-mail messages are stored in the hMailServer data directory (normally C:\Program Files\hMailServer\Data). The database contains "index lists" of these messages. For example, the path to the message file, the size of the message, message flags and recipients of the message is stored in the database.

are you sure it is a performance issue with the database ?
___________________________________________________________end of the line

User avatar
SorenR
Senior user
Senior user
Posts: 4169
Joined: 2006-08-21 15:38
Location: Denmark

Re: Multiple Instances on one DB Server

Post by SorenR » 2021-01-18 17:08

reesebruen wrote:
2021-01-18 13:26
Hi there,

I have a question where I couldn't find an answer yet.

Currently I'm running an instance of hMailserver on the embedded CE database. Now as I get quite a bit of emails a day, I would like
to move to a MySQL DB (at amazon RDS) where I expect a better performance.
For a single instance this works well.

Now my thoughts are, is it possible to have multiple hMailserver instances running on different servers, where both connect to the same IP?
Does anyone has experience on this? Or will it mess up the installation if you have multiple instances connecting to the same DB server?

Thanks for your help!

Cheers
Yes, you can have multiple servers (one hMailServer instance per server core or virtual server) connect to one common external MySQL server. Care must be taken to assure each hMailServer connect to each their own database during installation.
SørenR.

Algorithm (noun.)
Word used by programmers when they do not want to explain what they did.

User avatar
SorenR
Senior user
Senior user
Posts: 4169
Joined: 2006-08-21 15:38
Location: Denmark

Re: Multiple Instances on one DB Server

Post by SorenR » 2021-01-18 17:48

Just to clarify...

My primary hmailserver (Windows 2003 R2 Server) is also my MySQL database server and my secondary hmailserver (Windows 2019 Essential) is my Backup-MX. Both servers currently connect to the MySQL server instance on my primary server. So, two mailservers and one database server.
SørenR.

Algorithm (noun.)
Word used by programmers when they do not want to explain what they did.

User avatar
mattg
Moderator
Moderator
Posts: 21257
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Multiple Instances on one DB Server

Post by mattg » 2021-01-19 00:10

SorenR wrote:
2021-01-18 17:48
Just to clarify...

My primary hmailserver (Windows 2003 R2 Server) is also my MySQL database server and my secondary hmailserver (Windows 2019 Essential) is my Backup-MX. Both servers currently connect to the MySQL server instance on my primary server. So, two mailservers and one database server.
And ONE common file store
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

mikedibella
Senior user
Senior user
Posts: 384
Joined: 2016-12-08 02:21

Re: Multiple Instances on one DB Server

Post by mikedibella » 2021-01-19 01:31

mattg wrote:
2021-01-19 00:10
And ONE common file store
So, for a high-availability configuration, you have separate MTA process instances running on discrete nodes, with a common DBMS hosting a discrete database for each node, but with both nodes sharing common, networked, file-storage?

Couple of questions:

1. If a message is delivered to NodeA, how does NodeB discover this message so that the message can be retrieved via NodeB's client-access protocols? Does a scheduled task triggering the Data Synchronizer handle that task?

2. Does this architecture implement a single Queue? If an outbound message is delivered from an MUA to NodeB, and NodeB subsequently goes down, does NodeA see the Queued outbound message and relay it?

User avatar
SorenR
Senior user
Senior user
Posts: 4169
Joined: 2006-08-21 15:38
Location: Denmark

Re: Multiple Instances on one DB Server

Post by SorenR » 2021-01-19 02:13

mattg wrote:
2021-01-19 00:10
SorenR wrote:
2021-01-18 17:48
Just to clarify...

My primary hmailserver (Windows 2003 R2 Server) is also my MySQL database server and my secondary hmailserver (Windows 2019 Essential) is my Backup-MX. Both servers currently connect to the MySQL server instance on my primary server. So, two mailservers and one database server.
And ONE common file store
Nope...

Each server use their own harddrive for logging and mail storage. They are NOT mirrored. One is mailserver and the other is relay/backup-mx.

Now that would be a challenge... Making a hMailServer cluster.

hMailServer ==> fail-over clustering (two servers plus "witness")

Database ==> built-in clustering (a couple of Linux servers to do the dirty work)

File store ==> SAN ... $$$

Main issue would be to make the IP addresses shift during fail-over so active hMailServer server keep FQDN & IP address in DNS and allottet firewall slots.
SørenR.

Algorithm (noun.)
Word used by programmers when they do not want to explain what they did.

User avatar
SorenR
Senior user
Senior user
Posts: 4169
Joined: 2006-08-21 15:38
Location: Denmark

Re: Multiple Instances on one DB Server

Post by SorenR » 2021-01-19 02:21

mikedibella wrote:
2021-01-19 01:31
mattg wrote:
2021-01-19 00:10
And ONE common file store
So, for a high-availability configuration, you have separate MTA process instances running on discrete nodes, with a common DBMS hosting a discrete database for each node, but with both nodes sharing common, networked, file-storage?

Couple of questions:

1. If a message is delivered to NodeA, how does NodeB discover this message so that the message can be retrieved via NodeB's client-access protocols? Does a scheduled task triggering the Data Synchronizer handle that task?

2. Does this architecture implement a single Queue? If an outbound message is delivered from an MUA to NodeB, and NodeB subsequently goes down, does NodeA see the Queued outbound message and relay it?
It's much more complicated and it needs to happen in under 1 minute :mrgreen:

https://www.youtube.com/watch?v=lJLk3yPRV1w
SørenR.

Algorithm (noun.)
Word used by programmers when they do not want to explain what they did.

mikedibella
Senior user
Senior user
Posts: 384
Joined: 2016-12-08 02:21

Re: Multiple Instances on one DB Server

Post by mikedibella » 2021-01-19 02:51

I use two metrics for planning high availability.

One is the Recovery Time Objective, or RTO, which specifies the tolerable amount of time the service can be unavailable. An RTO of 5 minutes, for instance, specifies that it is tolerable for a failure to result in 5 minutes of downtime before recovery. For many architectures, RTO is a function of monitoring frequency. Failover to a redundant unit cannot happen faster than the heath of the services or service interfaces are polled for operational state.

The other is Recovery Point Objective, or RPO. RPO specifies the tolerable amount of data loss that the recovered service projects. An RPO of 1 hour means that it is tolerable for recovery to commence from a point in time 1 hour prior to the resumption of service.

So, you could have an RTO of 5 minutes based on polling the interfaces that often and failing over to a redundant unit when an anomaly is detected. Separately, you can be synchronizing data between nodes hourly, which means that when the redundant unit is promoted, the projected data state could be up to one hour old.

User avatar
SorenR
Senior user
Senior user
Posts: 4169
Joined: 2006-08-21 15:38
Location: Denmark

Re: Multiple Instances on one DB Server

Post by SorenR » 2021-01-19 03:29

mikedibella wrote:
2021-01-19 02:51
I use two metrics for planning high availability.

One is the Recovery Time Objective, or RTO, which specifies the tolerable amount of time the service can be unavailable. An RTO of 5 minutes, for instance, specifies that it is tolerable for a failure to result in 5 minutes of downtime before recovery. For many architectures, RTO is a function of monitoring frequency. Failover to a redundant unit cannot happen faster than the heath of the services or service interfaces are polled for operational state.

The other is Recovery Point Objective, or RPO. RPO specifies the tolerable amount of data loss that the recovered service projects. An RPO of 1 hour means that it is tolerable for recovery to commence from a point in time 1 hour prior to the resumption of service.

So, you could have an RTO of 5 minutes based on polling the interfaces that often and failing over to a redundant unit when an anomaly is detected. Separately, you can be synchronizing data between nodes hourly, which means that when the redundant unit is promoted, the projected data state could be up to one hour old.
We installed a mediation, rating & billing system at iBasis back in 1999/2000 on some SUN Enterprise servers (IIRC 7 or 8 servers) with Oracle DB and our system. IIRC we used Veritas, anyways it was spread over four sites in two states on the east coast. Average fail-over was expected to be less than 3 minutes including IP address migration to the alotted primarys... It was less than 3 minutes. I know, cause I accidentally made it happen once. :oops:

Which reminds me that the single worst word an IT tech kan say is ... oops!
SørenR.

Algorithm (noun.)
Word used by programmers when they do not want to explain what they did.

mikedibella
Senior user
Senior user
Posts: 384
Joined: 2016-12-08 02:21

Re: Multiple Instances on one DB Server

Post by mikedibella » 2021-01-19 03:40

SorenR wrote:
2021-01-19 02:13
Now that would be a challenge... Making a hMailServer cluster.

hMailServer ==> fail-over clustering (two servers plus "witness")

Database ==> built-in clustering (a couple of Linux servers to do the dirty work)

File store ==> SAN ... $$$

Main issue would be to make the IP addresses shift during fail-over so active hMailServer server keep FQDN & IP address in DNS and allottet firewall slots.
Thinking...

Two hMailServer instances running on separate guests, one primary, one redundant.

All configuration done to primary. Redundant unit has identical configuration to primary. Configurations synchronized using backup/restore script scheduled task running at RPO frequency.

Database server separate from HMS instances, implemented using vendor-specific fault tolerance.

Single shared file hierarchy on a NAS with vendor-specific fault tolerance. Data Directory Synchronizer runs as scheduled task RPO frequency. (Does DDS require service stop/pause?)

Pair of front-end LTMs (F5 BIG-IP, Citrix Netscaler, etc) publishing SMTP, PO3, and IMAP to primary with all-or-none failover to redundant based on consensus monitoring of interfaces. Monitors poll at RTO frequency.

User avatar
SorenR
Senior user
Senior user
Posts: 4169
Joined: 2006-08-21 15:38
Location: Denmark

Re: Multiple Instances on one DB Server

Post by SorenR » 2021-01-19 03:57

mikedibella wrote:
2021-01-19 03:40
Thinking...

Two hMailServer instances running on separate guests, one primary, one redundant.

All configuration done to primary. Redundant unit has identical configuration to primary. Configurations synchronized using backup/restore script scheduled task running at RPO frequency.
Only thing that needs to be sync'd is hmailserver.ini - all other settings are in the database
mikedibella wrote:
2021-01-19 03:40
Database server separate from HMS instances, implemented using vendor-specific fault tolerance.
MySQL with MySQL Mirroring (look it up) on each host keep databases in sync.
mikedibella wrote:
2021-01-19 03:40
Single shared file hierarchy on a NAS with vendor-specific fault tolerance. Data Directory Synchronizer runs as scheduled task RPO frequency. (Does DDS require service stop/pause?)
No need for DDS if databases are kept in sync.
mikedibella wrote:
2021-01-19 03:40
Pair of front-end LTMs (F5 BIG-IP, Citrix Netscaler, etc) publishing SMTP, PO3, and IMAP to primary with all-or-none failover to redundant based on consensus monitoring of interfaces. Monitors poll at RTO frequency.
Fail-over would be script on the sleeping server decommisioning IP address from dead server (booting it) to and assigning the IP address to local server and then start hMailServer.
When dead server return from boot it will detect other server is active and go into a wait loop.
SørenR.

Algorithm (noun.)
Word used by programmers when they do not want to explain what they did.

mikedibella
Senior user
Senior user
Posts: 384
Joined: 2016-12-08 02:21

Re: Multiple Instances on one DB Server

Post by mikedibella » 2021-01-19 04:03

SorenR wrote:
2021-01-19 03:57
Fail-over would be script on the sleeping server decommisioning IP address from dead server (booting it) to and assigning the IP address to local server and then start hMailServer.
When dead server return from boot it will detect other server is active and go into a wait loop.
One possible way to do this is by multi-homing the HMS instances. All sync/housekeeping via the "backside" always-up interface. SMTP/POP3/IMAP all bound to the "frontside" interface. Both nodes have same IP on frontside; only one frontside interface up at a time. Monitoring via backside to handle election. Each node responsible for self promotion/demotion based on election results.

User avatar
SorenR
Senior user
Senior user
Posts: 4169
Joined: 2006-08-21 15:38
Location: Denmark

Re: Multiple Instances on one DB Server

Post by SorenR » 2021-01-19 04:19

mikedibella wrote:
2021-01-19 04:03
SorenR wrote:
2021-01-19 03:57
Fail-over would be script on the sleeping server decommisioning IP address from dead server (booting it) to and assigning the IP address to local server and then start hMailServer.
When dead server return from boot it will detect other server is active and go into a wait loop.
One possible way to do this is by multi-homing the HMS instances. All sync/housekeeping via the "backside" always-up interface. SMTP/POP3/IMAP all bound to the "frontside" interface. Both nodes have same IP on frontside; only one frontside interface up at a time. Monitoring via backside to handle election. Each node responsible for self promotion/demotion based on election results.
Hmm... Windows tend to hang, not die... Would need something like VMWare.

vSphere Fault Tolerance ??
Fault Tolerance avoids "split-brain" situations, which can lead to two active copies of a virtual machine after recovery from a failure. Atomic file locking on shared storage is used to coordinate failover so that only one side continues running as the Primary VM and a new Secondary VM is respawned automatically.
SørenR.

Algorithm (noun.)
Word used by programmers when they do not want to explain what they did.

Post Reply