Basic IMAP logging option

Use this forum if you want to suggest a new feature to hMailServer. Before posting, please search the forum to confirm that it has not already been suggested.
Post Reply
SemperFidelis
New user
New user
Posts: 8
Joined: 2015-03-16 20:40

Basic IMAP logging option

Post by SemperFidelis » 2021-09-23 00:04

Hi team,

When enabling IMAP logging, we aim to log in priority sensitive security information
like the usernames and possible authentication issues/hacking attempts
which can occur during IMAP sessions
(otherwise we are blind regarding all the IMAP hacking attempts).

But alas, when enabling IMAP logging,
we are also overloading the log file with millions of lines related to FETCH command (see below) for each existing (already seen) message,
especially for users having huge mailboxes.

And these millions of FETCH commands can be repeated every 10 minutes
when the IMAP E-mail client queries again hmailServer for new message status
(like with Thunderbird). See an example below.

As a result, in order to keep security logging while not getting Gigabytes of daily logs,
can't we try to introduce a "basic" IMAP logging option ?

So only the crucial security information about IMAP sessions could continue to be logged,
while skipping millions of useless fetching information.

Thanks very much for your advice on this matter.

Example of useless IMAP logging:

"IMAPD" 5436 42039 "2021-09-22 11:56:11.361" "91.14.82.x" "SENT: * 144575 FETCH (UID 144575 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.362" "91.14.82.x" "SENT: * 144576 FETCH (UID 144576 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.362" "91.14.82.x" "SENT: * 144577 FETCH (UID 144577 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.363" "91.14.82.x" "SENT: * 144578 FETCH (UID 144578 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.363" "91.14.82.x" "SENT: * 144579 FETCH (UID 144579 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.364" "91.14.82.x" "SENT: * 144580 FETCH (UID 144580 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.364" "91.14.82.x" "SENT: * 144581 FETCH (UID 144581 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.365" "91.14.82.x" "SENT: * 144582 FETCH (UID 144582 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.365" "91.14.82.x" "SENT: * 144583 FETCH (UID 144583 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.366" "91.14.82.x" "SENT: * 144584 FETCH (UID 144584 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.366" "91.14.82.x" "SENT: * 144585 FETCH (UID 144585 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.367" "91.14.82.x" "SENT: * 144586 FETCH (UID 144586 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.367" "91.14.82.x" "SENT: * 144587 FETCH (UID 144587 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.368" "91.14.82.x" "SENT: * 144588 FETCH (UID 144588 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.368" "91.14.82.x" "SENT: * 144589 FETCH (UID 144589 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.369" "91.14.82.x" "SENT: * 144590 FETCH (UID 144590 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.369" "91.14.82.x" "SENT: * 144591 FETCH (UID 144591 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.370" "91.14.82.x" "SENT: * 144592 FETCH (UID 144592 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.370" "91.14.82.x" "SENT: * 144593 FETCH (UID 144593 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.371" "91.14.82.x" "SENT: * 144594 FETCH (UID 144594 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.371" "91.14.82.x" "SENT: * 144595 FETCH (UID 144595 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.371" "91.14.82.x" "SENT: * 144596 FETCH (UID 144596 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.372" "91.14.82.x" "SENT: * 144597 FETCH (UID 144597 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.372" "91.14.82.x" "SENT: * 144598 FETCH (UID 144598 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.372" "91.14.82.x" "SENT: * 144599 FETCH (UID 144599 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.373" "91.14.82.x" "SENT: * 144600 FETCH (UID 144600 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.373" "91.14.82.x" "SENT: * 144601 FETCH (UID 144601 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.373" "91.14.82.x" "SENT: * 144602 FETCH (UID 144602 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.373" "91.14.82.x" "SENT: * 144603 FETCH (UID 144603 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.374" "91.14.82.x" "SENT: * 144604 FETCH (UID 144604 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.374" "91.14.82.x" "SENT: * 144605 FETCH (UID 144605 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.374" "91.14.82.x" "SENT: * 144606 FETCH (UID 144606 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.374" "91.14.82.x" "SENT: * 144607 FETCH (UID 144607 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.375" "91.14.82.x" "SENT: * 144608 FETCH (UID 144608 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.375" "91.14.82.x" "SENT: * 144609 FETCH (UID 144609 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.375" "91.14.82.x" "SENT: * 144610 FETCH (UID 144610 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.375" "91.14.82.x" "SENT: * 144611 FETCH (UID 144611 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.376" "91.14.82.x" "SENT: * 144612 FETCH (UID 144612 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.376" "91.14.82.x" "SENT: * 144613 FETCH (UID 144613 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.376" "91.14.82.x" "SENT: * 144614 FETCH (UID 144614 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.376" "91.14.82.x" "SENT: * 144615 FETCH (UID 144615 FLAGS (\Seen))"
"IMAPD" 5436 42039 "2021-09-22 11:56:11.377" "91.14.82.x" "SENT: * 144616 FETCH (UID 144616 FLAGS (\Seen))"

User avatar
mattg
Moderator
Moderator
Posts: 22435
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Basic IMAP logging option

Post by mattg » 2021-09-23 01:05

Yeah, I agree
IMAP logging is VERY verbose

I only turn it on to track down some hard to find issues

I do, however, create lots of custom logs, many built on RvdH build custom included events
viewtopic.php?f=10&t=30193&start=240#p232710

Which version of hMailserver do you run?
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

palinka
Senior user
Senior user
Posts: 4455
Joined: 2017-09-12 17:57

Re: Basic IMAP logging option

Post by palinka » 2021-09-23 01:39

SemperFidelis wrote:
2021-09-23 00:04
Hi team,

When enabling IMAP logging, we aim to log in priority sensitive security information
like the usernames and possible authentication issues/hacking attempts
which can occur during IMAP sessions
(otherwise we are blind regarding all the IMAP hacking attempts).
Hey, jarhead! Fellow leatherneck here, 1987-91. Good to meet you.

Matt is right. Those custom events are just what's needed. In particular, OnClientLogon will get you squared away most riki tik. :wink:

You can filter by port, then by unsuccessful logon to get what you're looking for. Write results to the event log or to a custom log.

But, as Matt said, you need to install RvdH's custom version, or the 64 bit 5.7 alpha, which really should be beta or even further along by now. I've been running that for a while, but my server is just a Tonka toy with very low volume. Anyway, you have a couple of choices.

Post Reply