STARTTLS feedback?

Use this forum if you want to discuss a problem or ask a question related to a hMailServer beta release.
joshiewa
New user
New user
Posts: 2
Joined: 2014-08-13 03:25

Re: STARTTLS feedback?

Post by joshiewa » 2014-08-14 01:24

Great, thanks for that... love the product and will be donating.

Out of interest, is there an ETA for starttls to be in a production build?

User avatar
mattg
Moderator
Moderator
Posts: 22435
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: STARTTLS feedback?

Post by mattg » 2014-08-14 01:37

In this thread >> viewtopic.php?f=10&t=24921
Martin says should be ready by October
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
martin
Developer
Developer
Posts: 6846
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: STARTTLS feedback?

Post by martin » 2014-08-14 10:27

That was a reply to a user who planned to start using it in October. It is already feature complete and I will put it up as beta today. A production release in a few weeks unless lots of issues are found.

prisma
Senior user
Senior user
Posts: 325
Joined: 2010-07-09 13:16

Re: STARTTLS feedback?

Post by prisma » 2014-08-14 11:18

Is B2052 OK for testing or are some changes in queue I should wait for?
... OK, I see you wrote: It's feature complete. Stupid question...

User avatar
martin
Developer
Developer
Posts: 6846
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: STARTTLS feedback?

Post by martin » 2014-08-14 20:02

The beta is available on the download page now.

percepts
Senior user
Senior user
Posts: 5282
Joined: 2009-10-20 16:33
Location: Sceptred Isle

Re: STARTTLS feedback?

Post by percepts » 2014-08-14 21:19

have you upgraded PHPWebAdmin for all this STARTTLS stuff.
I just installed 5.5-B2053 and installed what was in PHPWebAdmin and there is nothing for the new starttls in IP-Ranges or TCP/IP ports

version check file says

define('REQUIRED_VERSION', '5.5-B2053');

but all the files seem to have been created on 2014-06-24 which I figure is before you made all these changes.

User avatar
martin
Developer
Developer
Posts: 6846
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: STARTTLS feedback?

Post by martin » 2014-08-15 09:04

Correct, the WebAdmin is not updated yet. That will happen in the next build.

User avatar
martin
Developer
Developer
Posts: 6846
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: STARTTLS feedback?

Post by martin » 2014-08-15 10:03

I've commited the changes to WebAdmin. Have not have time to test it on my own server yet, but if you feel eager, brave and have a server to waste, you can download the binary from here:

https://build.hmailserver.com/

prisma
Senior user
Senior user
Posts: 325
Joined: 2010-07-09 13:16

Re: STARTTLS feedback?

Post by prisma » 2014-08-15 11:16

Question: What does Page "Security" Checkbox "Verify remote server SSL/TLS certificates" exactly configure?
Or is this checkbox for deactivation of ciphers below? I'm confused....

percepts
Senior user
Senior user
Posts: 5282
Joined: 2009-10-20 16:33
Location: Sceptred Isle

Re: STARTTLS feedback?

Post by percepts » 2014-08-15 12:56

5.5-B2054 PHPWebAdmin Tests

IP-Range ADD and Display panel OTHER is missing "require SSL/TLS for authentication"

SMTP / Routes missing new security option on dispaly and add panels

prisma
Senior user
Senior user
Posts: 325
Joined: 2010-07-09 13:16

Re: STARTTLS feedback?

Post by prisma » 2014-08-15 13:59

Following was very helpful for testing but also for every end-email-user to check which encrypted or not encrypted way his email has taken:

Add header info:

Code: Select all

by mx.mydomain.com with ESMTPS (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
or

Code: Select all

by mx.mydomain.com with ESMTPSA (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
It thought Bill already added at least "ESMTPS"/"ESMTPSA", didn't he? This was good enough for an experimental build. This info is missing in the moment, isn't it?

User avatar
RvdH
Senior user
Senior user
Posts: 3231
Joined: 2008-06-27 14:42
Location: The Netherlands

Re: STARTTLS feedback?

Post by RvdH » 2014-08-15 16:44

Code: Select all

"Performing SSL/TLS handshake for session [NR]. Verify certificate: False, Verify mode: 0, Expected remote: "
What does the above exactly say?
CIDR to RegEx: d-fault.nl/cidrtoregex
DNS Lookup: d-fault.nl/dnstools
DKIM Generator: d-fault.nl/dkimgenerator
DNSBL Lookup: d-fault.nl/dnsbllookup
GEOIP Lookup: d-fault.nl/geoiplookup

User avatar
martin
Developer
Developer
Posts: 6846
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: STARTTLS feedback?

Post by martin » 2014-08-15 17:34

prisma wrote:What does Page "Security" Checkbox "Verify remote server SSL/TLS certificates" exactly configure?
I've removed this setting from the UI now. What it did was to tell hMailServer whether or not it should verify certificates during SSL Handshake, a global setting overriding everything else. So even if you have selected STARTTLS (Required), you could disable the certificate verification-step of the handshake. This setting is really not useful anymore after the switch to Crypto API, so I've removed it.
percepts wrote:IP-Range ADD and Display panel OTHER is missing "require SSL/TLS for authentication"
SMTP / Routes missing new security option on dispaly and add panels
I've added to these two. But I need to run some overall testing of WebAdmin. (I dislike this part of hMailServer, it's so confusing two have two different frontends to manage the software. Maybe I should bundle PHP and a small web server, refine the Web UI to look/behave good and ditch Administrator).
prisma wrote:It thought Bill already added at least "ESMTPS"/"ESMTPSA", didn't he? This was good enough for an experimental build. This info is missing in the moment, isn't it?
Correct. The implementation is not based on Bills changes.
RvdH wrote:What does the above exactly say?
  • Performing SSL/TLS handshake for session - This basically means that a SSL or TLS handshake has been initiated.
  • Verify certificate: False - This means that the remote peer's certificate will not be verified.
  • Verify mode: 0 - You can ignore this. It's redundant and will be removed in the next build (just removed it). It's basically the verification parameter for OpenSSL, but that's hard-coded depending on "Verify Certificate" so no need to include that value.
  • Expected remote: - This is short for "expected remote hostname". If you deliver to some gmail server it may be something like 'gmail-smtp-in.l.google.com'. I've done two changes: 1) Replaced text with "Expected remote hostname", 2) Removed this part of the log if no specific host name is epected.

User avatar
martin
Developer
Developer
Posts: 6846
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: STARTTLS feedback?

Post by martin » 2014-08-15 17:45

(Note that there currently is an issue with this build when it comes to sending larger messages, it seems. I will publish a new beta later today).

percepts
Senior user
Senior user
Posts: 5282
Joined: 2009-10-20 16:33
Location: Sceptred Isle

Re: STARTTLS feedback?

Post by percepts » 2014-08-15 17:59

martin wrote:
percepts wrote:IP-Range ADD and Display panel OTHER is missing "require SSL/TLS for authentication"
SMTP / Routes missing new security option on dispaly and add panels
I've added to these two. But I need to run some overall testing of WebAdmin. (I dislike this part of hMailServer, it's so confusing two have two different frontends to manage the software. Maybe I should bundle PHP and a small web server, refine the Web UI to look/behave good and ditch Administrator).
the C# admin GUI is far superior IMO. Much nicer to work with. I wouldn't like to see it go. The Web version is a bit lacking in some areas, no rule sort order for one. But I guess it's a must for users to be able to change out of office message or domain owners to be able to add/delete/change user settings. Main mailserver admin can always remote connect for full functionality. i.e. Most of web admin probably isn't required or at most by very few.

User avatar
RvdH
Senior user
Senior user
Posts: 3231
Joined: 2008-06-27 14:42
Location: The Netherlands

Re: STARTTLS feedback?

Post by RvdH » 2014-08-15 18:03

percepts wrote:
martin wrote:
percepts wrote:IP-Range ADD and Display panel OTHER is missing "require SSL/TLS for authentication"
SMTP / Routes missing new security option on dispaly and add panels
I've added to these two. But I need to run some overall testing of WebAdmin. (I dislike this part of hMailServer, it's so confusing two have two different frontends to manage the software. Maybe I should bundle PHP and a small web server, refine the Web UI to look/behave good and ditch Administrator).
the C# admin GUI is far superior IMO. Much nicer to work with. I wouldn't like to see it go. The Web version is a bit lacking in some areas, no rule sort order for one. But I guess it's a must for users to be able to change out of office message or domain owners to be able to add/delete/change user settings. Main mailserver admin can always remote connect for full functionality. i.e. Most of web admin probably isn't required or at most by very few.
I agree...never really used the webadmin
CIDR to RegEx: d-fault.nl/cidrtoregex
DNS Lookup: d-fault.nl/dnstools
DKIM Generator: d-fault.nl/dkimgenerator
DNSBL Lookup: d-fault.nl/dnsbllookup
GEOIP Lookup: d-fault.nl/geoiplookup

User avatar
martin
Developer
Developer
Posts: 6846
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: STARTTLS feedback?

Post by martin » 2014-08-15 19:35

percepts wrote:the C# admin GUI is far superior IMO. Much nicer to work with. I wouldn't like to see it go.
Maybe I was a bit unclear, I'm not talking about ditching hMailServer Administrator and use WebAdmin in its current form. If hMailServer Administrator is ever phased out, same or better functionality and usability will exist in WebAdmin. This is far down the road though and I probably will not have time to do it.

A lot has happened to web development since I wrote the WebAdmin. I think that today its possible to write a web admin interface which is on par (and better) than the current C#-administrator. The WebAdmin in its current form is a hack. It's really easy to add new checkboxes and so on, but it's sluggish to use, ugly and lack functionality.

percepts
Senior user
Senior user
Posts: 5282
Joined: 2009-10-20 16:33
Location: Sceptred Isle

Re: STARTTLS feedback?

Post by percepts » 2014-08-15 19:53

well yes, with html5, ajax and jquery it could be a whole lot better but thats another learning curve.

percepts
Senior user
Senior user
Posts: 5282
Joined: 2009-10-20 16:33
Location: Sceptred Isle

Re: STARTTLS feedback?

Post by percepts » 2014-08-15 20:54

And a web only admin interface presumes that user has a webserver available or knows how and wants to install one.
Then there is the com api remote problem which people have for whatever reason, they may not want a webserver on same box as hmailserver. And they may be using more than one implementation of hmail all of which would require a different domain to administer. It could be a disaster for achieving a smooth install.

User avatar
martin
Developer
Developer
Posts: 6846
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: STARTTLS feedback?

Post by martin » 2014-08-15 21:31

I see what you're saying and don't worry, I won't just throw Admin out with nothing to replace it smoothly. :)

If hMailServer would bundle web admin as only interface, it would be sane to use Microsofts Owin to host an in-proc light-weight web server on some special port. Owin+Katana allows you to expose a web server from within a single executable. There's no need to deploy a full-blown IIS any more. From an end users perspective it would be possible to get roughly the same ease of installation as today using this technology. The company where I have my day-time job ships software based on these and I've spent quite some time experimenting with it myself. It's really smooth, compared to the pain of using IIS.

(Owin+Katana is the future of Microsofts web stack and the next version of ASP.NET will be based off it. I've seen some pretty cool presentations where they create a web site using ASP.NET and then xcopy-deploy-and-run it on a Mac borrowed from the audience. Impressive stuff.)

percepts
Senior user
Senior user
Posts: 5282
Joined: 2009-10-20 16:33
Location: Sceptred Isle

Re: STARTTLS feedback?

Post by percepts » 2014-08-15 22:27

OMG the wheel has been re-invented again. Well I suppose they have to keep their revenue stream going somehow.

User avatar
mattg
Moderator
Moderator
Posts: 22435
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: STARTTLS feedback?

Post by mattg » 2014-08-16 01:01

RvdH wrote:
percepts wrote:The Web version is a bit lacking in some areas, no rule sort order for one. But I guess it's a must for users to be able to change out of office message or domain owners to be able to add/delete/change user settings. Main mailserver admin can always remote connect for full functionality. i.e. Most of web admin probably isn't required or at most by very few.
I agree...never really used the webadmin
I use it a lot.

There is no easy way to allow users to change their own settings without it (rules, forwards, out-of-office)
It saves me doing the work for them.

I also use it, when I am away from my computer to 'manage' the mail server.

I like Martin's idea of a single (but better than what we have) GUI interface.
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
RvdH
Senior user
Senior user
Posts: 3231
Joined: 2008-06-27 14:42
Location: The Netherlands

Re: STARTTLS feedback?

Post by RvdH » 2014-08-16 12:00

Something is wrong with the values reported in the status tab, 'Number of open sessions'
On my install it always reports 2 SMTP session, 1 POP3 session en 2 IMAP sessions but if i check open ports on system i only see 1 IMAP session active

even after a restart is shows 2 SMTP session, 1 POP3 session en 1 IMAP sessions active immediately
CIDR to RegEx: d-fault.nl/cidrtoregex
DNS Lookup: d-fault.nl/dnstools
DKIM Generator: d-fault.nl/dkimgenerator
DNSBL Lookup: d-fault.nl/dnsbllookup
GEOIP Lookup: d-fault.nl/geoiplookup

User avatar
mattg
Moderator
Moderator
Posts: 22435
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: STARTTLS feedback?

Post by mattg » 2014-08-16 13:57

martin wrote:
RvdH wrote:What does the above exactly say?
  • Performing SSL/TLS handshake for session - This basically means that a SSL or TLS handshake has been initiated.
  • Verify certificate: False - This means that the remote peer's certificate will not be verified.
  • Verify mode: 0 - You can ignore this. It's redundant and will be removed in the next build (just removed it). It's basically the verification parameter for OpenSSL, but that's hard-coded depending on "Verify Certificate" so no need to include that value.
  • Expected remote: - This is short for "expected remote hostname". If you deliver to some gmail server it may be something like 'gmail-smtp-in.l.google.com'. I've done two changes: 1) Replaced text with "Expected remote hostname", 2) Removed this part of the log if no specific host name is epected.
Is the verify certificate currently enforcable?
Can I somehow turn on verification of the remote peer's certificate?
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
martin
Developer
Developer
Posts: 6846
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: STARTTLS feedback?

Post by martin » 2014-08-17 15:36

> Is the verify certificate currently enforcable?

If you're adding a route or specifying a SMTP relayer, and select connection security STARTTLS (Required) then certificate verification will be enforced.

User avatar
mattg
Moderator
Moderator
Posts: 22435
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: STARTTLS feedback?

Post by mattg » 2014-08-17 16:32

What I'd really like to do, is log all verifications of certificates, without actually requiring verification for mail delivery
I'd like to see how big an issue verification of certificates is for me.

I was thinking verified certificates for incoming mail on port 25, without requiring all incoming mail to be StartTLS, may also be a reason to skip greylisting
skip greylisting on SPF pass is a good start except that there are literally thousands of domains with a SPF record that ends in +all, and also hmailserver in default mode passes soft fails, ie those with an spf record that ends in ~all

In Bill's alpha builds, he had set via INI settings so that I could set a default spf record, and could effectively fail those with a +all
Did these INI settings for spf make it into the Beta build?
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
martin
Developer
Developer
Posts: 6846
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: STARTTLS feedback?

Post by martin » 2014-08-17 19:02

If you use STARTTLS (Optional) it will do basically that. hMailServer will attempt to use STARTTLS whenever possible, and will verify certificates, but if certificate verification fails it will just log a message and pretend everything is good. This log message is currently part of the Debug log though, which may be a problem depending on how much logs you have.

The different types of connection security is described here:
https://hmailserver.com/documentation/l ... n_security
I was thinking verified certificates for incoming mail on port 25
I'm not sure what you mean here though. Spammers may be less likely to use STARTTLS - is this what you're saying?

I don't believe the settings for default spf record is included unless they were done some time ago and he has included them himself. I was given access to the source code little over a week ago so haven't digged into it yet.

User avatar
mattg
Moderator
Moderator
Posts: 22435
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: STARTTLS feedback?

Post by mattg » 2014-08-18 01:19

martin wrote:If you use STARTTLS (Optional) it will do basically that. hMailServer will attempt to use STARTTLS whenever possible, and will verify certificates, but if certificate verification fails it will just log a message and pretend everything is good. This log message is currently part of the Debug log though, which may be a problem depending on how much logs you have.
I have lots of these in my logs

Code: Select all

"SMTPD"	14544	1946	"2014-08-18 09:00:16.214"	"xxx.xxx.xxx.xxx"	"SENT: 220 Ready to start TLS"
"DEBUG"	14544	"2014-08-18 09:00:16.215"	"Performing SSL/TLS handshake for session 1946. Verify certificate: False"
Doesn't that mean that the certificate isn't verified?
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
martin
Developer
Developer
Posts: 6846
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: STARTTLS feedback?

Post by martin » 2014-08-18 09:53

I suspect what you are after is not (yet) implemented in hMailServer. I've added this to the documentation:
As a part of the SSL/TLS handshake, hMailServer will verify that the server it connects to has a correct certificate. This includes several things, for example: ....
Note that hMailServer does not verify remote client certificates - it only verifies the certificates of servers it connects to. If a end-user connects to hMailServer using for example Thunderbird or Outlook, he does not have to provide a certificate. A remote server which delivers a mail to hMailServer does not have to provide a certificate either. In these scenarios - where a client is connecting to hMailServer, hMailServer will present a certificate which will be used to encrypt the session.
This is similar to the behavior you see when you use your normal web browser. If you go to https://www.google.com, you don't have to present a client certificate.

User avatar
mattg
Moderator
Moderator
Posts: 22435
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: STARTTLS feedback?

Post by mattg » 2014-08-18 11:04

To be honest, that's what I was expecting.
I guess I was just confused with what was logged
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
martin
Developer
Developer
Posts: 6846
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: STARTTLS feedback?

Post by martin » 2014-08-18 11:26

I see. Those entries appears to be related to a STARTTLS initiated by a client, maybe Thunderbird, a webmail or whatever. hMailServer could verify the certificate in those cases also, but I suspect it's quite rare that it will be useful.

prisma
Senior user
Senior user
Posts: 325
Joined: 2010-07-09 13:16

Re: STARTTLS feedback?

Post by prisma » 2014-08-18 11:47

martin wrote: Correct. The implementation is not based on Bills changes.
Would it be very complex to add it? All mailservers I know do add these header information. It's very useful and seems to be good practise.
martin wrote: I see. Those entries appears to be related to a STARTTLS initiated by a client, maybe Thunderbird, a webmail or whatever. hMailServer could verify the certificate in those cases also, but I suspect it's quite rare that it will be useful.
You're talking about "client certificate authentication". This an user authentication method. It's used to replace the password or as second factor. It's used within enterprise security. But I think this is the step after the step after DANE implementation :)

User avatar
mattg
Moderator
Moderator
Posts: 22435
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: STARTTLS feedback?

Post by mattg » 2014-08-18 14:05

martin wrote:I see. Those entries appears to be related to a STARTTLS initiated by a client, maybe Thunderbird, a webmail or whatever. hMailServer could verify the certificate in those cases also, but I suspect it's quite rare that it will be useful.
It was actually mail coming from another mail server.

I can send you more of the logs via email if you like.

This wasn't from gmail, but the same things happens with mail from gmail to an address hosted on my hmailserver.
It may well be that there is no certificate to verify (what I expect), but it seems that hMailserver is considering verification.
Perhaps verification of my certificate...
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

sckramer2
Normal user
Normal user
Posts: 134
Joined: 2009-07-31 21:50

Re: STARTTLS feedback?

Post by sckramer2 » 2014-08-18 18:16

Would adding a counter for successfully/unsuccessfully encrypted messages be worthwhile on the status page?

User avatar
martin
Developer
Developer
Posts: 6846
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: STARTTLS feedback?

Post by martin » 2014-08-18 19:00

mattg wrote:It was actually mail coming from another mail server.
When I wrote client, I meant some 3rd party opening a connection to hMailServer, rather than hMailServer opening a connection to another domain. The relevant part of the documentation is this:
Note that hMailServer does not verify remote client certificates - it only verifies the certificates of servers it connects to
If Gmail/Thunderbird/Outlook/SomeOtherSmtpServer connects to your hMailServer, hMailServer will not attempt to validate that clients certificate. When this happens, "Verify certificate: False" will be logged. This does not mean that hMailServer has failed to verify the certificate - it means that hMailServer will not attempt to verify the certificate.

prisma
Senior user
Senior user
Posts: 325
Joined: 2010-07-09 13:16

Re: STARTTLS feedback?

Post by prisma » 2014-08-19 12:10

Very interesting conversation :) But I think this should be clear know ;) (just kidding) But please excuse if I insist:

Would it be very complex to get header info back? It seems really to be good practise. All mailservers I know behave like this, except hmailserver.

percepts
Senior user
Senior user
Posts: 5282
Joined: 2009-10-20 16:33
Location: Sceptred Isle

Re: STARTTLS feedback?

Post by percepts » 2014-08-19 17:39

testing starttls:

so far so good. I'm using "use starttls if available" everywhere on ports 25,110,143,587and ssl/tls on 465,993,995 and all seems to be working so far.

One thing I would like is to add port number to TCP/IP log message in smtpc and pop3

Code: Select all

"TCPIP"	5960	"2014-08-19 16:29:31.835"	"Connecting to 173.194.67.109..."
just so its obvious which port is being used.

One thing I don't get is "Require SSl/TLS for authentication" in Ip-Ranges. When should it be used? I thought that if you were using starttls or ssl/tls then authentication would be done securely anyway.

User avatar
martin
Developer
Developer
Posts: 6846
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: STARTTLS feedback?

Post by martin » 2014-08-19 19:55

prisma wrote:Would it be very complex to get header info back?
"Back" is the wrong term since it has never been there in hMailServer. None of the code Bill wrote for STARTTLS is part of hMailServer. I'll check what it takes to add it though.
percepts wrote:One thing I would like is to add port number to TCP/IP log message in smtpc and pop3
Changed the message to Connecting to ip:port...
percepts wrote:One thing I don't get is "Require SSl/TLS for authentication" in Ip-Ranges. When should it be used? I thought that if you were using starttls or ssl/tls then authentication would be done securely anyway.
For port 25, you want to use STARTTLS (Optional) because other mail servers might not support STARTTLS. But this also means that your own users may send email using this port without using STARTTLS or any form of encryption. You can select that option in the IP range if you want to force local users to always use STARTTLS or SSL/TLS.

percepts
Senior user
Senior user
Posts: 5282
Joined: 2009-10-20 16:33
Location: Sceptred Isle

Re: STARTTLS feedback?

Post by percepts » 2014-08-19 21:21

martin wrote:
percepts wrote:One thing I would like is to add port number to TCP/IP log message in smtpc and pop3
Changed the message to Connecting to ip:port...
Thanks,

Another thing that I think needs cleaning up is the INI file. Currently it doesn't show all the [Settings] options in the default install INI file. I know of at least a couple, there may be others as I don't know which of Bills have been included in current beta. It just means you know what you've got to play with without going looking for it in the documentation or forums and would provide greater clarity. They can all be commeneted out to start with I guess.

Code: Select all

[Settings]
; the default "PreferredHashAlgorithm" is 3
; 0 - None. Store passwords in clear text. This is not recommended.
; 1 - Blowfish. Store passwords encrypted using Blowfish. This is not recommended, since the password used for ;encryption is known. Hence, this is no more safe than option 0.
; 2 - MD5. Store passwords in MD5 hash. This is only recommended to preserve backwards compatibility if you have ;application which integrates with the hMailServer database.
;◦  - SHA256 - Store passwords in SHA256 hashes. This is currently the recommended option which gives the highest level of security.
;PreferredHashAlgorithm=3

; This key tells hmailserver to split SMTP/IMAP/POP into their own log files such as
; hmailserver_SMTP_2010-10-24.log hmailserver_IMAP_2010-10-24.log hmailserver_POP3_2010-10-24.log
; Default is to have all services logged together in 1 file.
;SepSvcLogs=1

User avatar
mattg
Moderator
Moderator
Posts: 22435
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: STARTTLS feedback?

Post by mattg » 2014-08-20 00:14

martin wrote:
percepts wrote:One thing I don't get is "Require SSl/TLS for authentication" in Ip-Ranges. When should it be used? I thought that if you were using starttls or ssl/tls then authentication would be done securely anyway.
For port 25, you want to use STARTTLS (Optional) because other mail servers might not support STARTTLS. But this also means that your own users may send email using this port without using STARTTLS or any form of encryption. You can select that option in the IP range if you want to force local users to always use STARTTLS or SSL/TLS.
I haven't tried, but just wondering if I could have 2 internet IP ranges.

1. allows external to local only, and doesn't require StartTLS
2. A higher priority, and allows all connections, but requires authentication AND StartTLS

Maybe I'll just try that and see...
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

percepts
Senior user
Senior user
Posts: 5282
Joined: 2009-10-20 16:33
Location: Sceptred Isle

Re: STARTTLS feedback?

Post by percepts » 2014-08-20 01:17

I have managed to send to hmail to gmail using starttls so I was getting secure server to server on port 25 which is good.

I also managed to get account external download from gmail on 995/ssl which I had never managed to get working before for some reason but seems to work now.

User avatar
RvdH
Senior user
Senior user
Posts: 3231
Joined: 2008-06-27 14:42
Location: The Netherlands

Re: STARTTLS feedback?

Post by RvdH » 2014-08-20 12:24

percepts wrote: Another thing that I think needs cleaning up is the INI file. Currently it doesn't show all the [Settings] options in the default install INI file. I know of at least a couple, there may be others as I don't know which of Bills have been included in current beta. It just means you know what you've got to play with without going looking for it in the documentation or forums and would provide greater clarity. They can all be commeneted out to start with I guess.

Code: Select all

[Settings]
; the default "PreferredHashAlgorithm" is 3
; 0 - None. Store passwords in clear text. This is not recommended.
; 1 - Blowfish. Store passwords encrypted using Blowfish. This is not recommended, since the password used for ;encryption is known. Hence, this is no more safe than option 0.
; 2 - MD5. Store passwords in MD5 hash. This is only recommended to preserve backwards compatibility if you have ;application which integrates with the hMailServer database.
;◦  - SHA256 - Store passwords in SHA256 hashes. This is currently the recommended option which gives the highest level of security.
;PreferredHashAlgorithm=3

; This key tells hmailserver to split SMTP/IMAP/POP into their own log files such as
; hmailserver_SMTP_2010-10-24.log hmailserver_IMAP_2010-10-24.log hmailserver_POP3_2010-10-24.log
; Default is to have all services logged together in 1 file.
;SepSvcLogs=1
https://www.hmailserver.com/documentati ... lesettings

These INI settings only apply for Bill's Experimental builds, not for the official STARTTLS beta's Martin now releases
CIDR to RegEx: d-fault.nl/cidrtoregex
DNS Lookup: d-fault.nl/dnstools
DKIM Generator: d-fault.nl/dkimgenerator
DNSBL Lookup: d-fault.nl/dnsbllookup
GEOIP Lookup: d-fault.nl/geoiplookup

percepts
Senior user
Senior user
Posts: 5282
Joined: 2009-10-20 16:33
Location: Sceptred Isle

Re: STARTTLS feedback?

Post by percepts » 2014-08-20 12:30

The two I have posted ARE IN 5.5. I don't know if any others have been included.

User avatar
SorenR
Senior user
Senior user
Posts: 6308
Joined: 2006-08-21 15:38
Location: Denmark

Re: STARTTLS feedback?

Post by SorenR » 2014-08-20 14:10

percepts wrote:The two I have posted ARE IN 5.5. I don't know if any others have been included.
Check source code on Github... The two settings you posted are included in 5.4.2-B1964 too.. I personally use SepSvcLogs=1...

Actually there is a ton of additional settings in 5.4.2 but I guess only <1% of all installations would use them.
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

percepts
Senior user
Senior user
Posts: 5282
Joined: 2009-10-20 16:33
Location: Sceptred Isle

Re: STARTTLS feedback?

Post by percepts » 2014-08-20 14:24

SorenR wrote:
percepts wrote:The two I have posted ARE IN 5.5. I don't know if any others have been included.
Check source code on Github... The two settings you posted are included in 5.4.2-B1964 too.. I personally use SepSvcLogs=1...

Actually there is a ton of additional settings in 5.4.2 but I guess only <1% of all installations would use them.
Yes I know but its confusing to know which should work and which shouldn't in current version compared to Bills version.

User avatar
SorenR
Senior user
Senior user
Posts: 6308
Joined: 2006-08-21 15:38
Location: Denmark

Re: STARTTLS feedback?

Post by SorenR » 2014-08-20 14:42

percepts wrote:
SorenR wrote:
percepts wrote:The two I have posted ARE IN 5.5. I don't know if any others have been included.
Check source code on Github... The two settings you posted are included in 5.4.2-B1964 too.. I personally use SepSvcLogs=1...

Actually there is a ton of additional settings in 5.4.2 but I guess only <1% of all installations would use them.
Yes I know but its confusing to know which should work and which shouldn't in current version compared to Bills version.
This should be the very latest on Github...
https://github.com/hmailserver/hmailser ... ttings.cpp

Search for ("Settings", there are 38 of them...
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

percepts
Senior user
Senior user
Posts: 5282
Joined: 2009-10-20 16:33
Location: Sceptred Isle

Re: STARTTLS feedback?

Post by percepts » 2014-08-20 15:27

SorenR wrote: Search for ("Settings", there are 38 of them...
but are they all implemented in the program rather than just loaded to memory and not used?

User avatar
SorenR
Senior user
Senior user
Posts: 6308
Joined: 2006-08-21 15:38
Location: Denmark

Re: STARTTLS feedback?

Post by SorenR » 2014-08-20 15:54

percepts wrote:
SorenR wrote: Search for ("Settings", there are 38 of them...
but are they all implemented in the program rather than just loaded to memory and not used?
It appears that only "GreylistingRecordExpirationInterval" is not referenced outside the inifile definition...
SørenR.

Woke is Marxism advancing through Maoist cultural revolution.

User avatar
martin
Developer
Developer
Posts: 6846
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: STARTTLS feedback?

Post by martin » 2014-08-20 15:55

Just want to add that the ini file settings are not tested, except for the basic ones (different directories, database login). I know Bill has added quite some settings to the ini file, but these new settings are not at all covered by tests. Since they are not tested at all in any consistent manner, I will not add these settings to the documentation.

I don't rely on stuff which there exists no automated tests for. Also, I don't feel that having settings in the ini-file is a good way way forward. hMailServer stores configuration in a database and that configuration can be accessed using the API. Settings in the ini file cannot be accessed or updated using the API and not auto-tested in a sane way. Stuff should be in the database, accessible via the API and changeable in the UI before I consider it "implemented".......

percepts
Senior user
Senior user
Posts: 5282
Joined: 2009-10-20 16:33
Location: Sceptred Isle

Re: STARTTLS feedback?

Post by percepts » 2014-08-20 16:15

Ah, well at least I know they are in the code, are untested and for all I know may disappear at any time. On the other hand they may be added to API anytime :mrgreen:

So for now use at own risk if you know what they are but we're not telling.

percepts
Senior user
Senior user
Posts: 5282
Joined: 2009-10-20 16:33
Location: Sceptred Isle

Re: STARTTLS feedback?

Post by percepts » 2014-08-26 14:17

hmailserver 5.5 B2063

gmx.co.uk doesn't want to play.

It looks like they will try to use STARTTLS but gives "SSL handshake with client failed".

I don't know if this is due to my self signed certificate (which works with gmail) or its a gmx thing or an hmail thing.
Mail does get sent but it just won't do it using STARTTLS to gmx.co.uk and falls back to ESMTP.
Or have I mis-interpreted STARTTLS (marked in Red) incorrectly as meaning gmx can receive using STARTTLS ?
And I'm not sure why SSL handshake is taking place if gmx hasn't indicated that it can do STARTTLS.

hmail is set to optionally use STARTTLS if availble on port 25.
"TCPIP" 5836 "2014-08-26 12:54:45.794" "Connecting to 213.165.67.115:25..."
"SMTPC" 7032 2102 "2014-08-26 12:54:45.919" "213.165.67.115" "RECEIVED: 220 gmx.net (mxgmx107) Nemesis ESMTP Service ready"
"SMTPC" 7032 2102 "2014-08-26 12:54:45.919" "213.165.67.115" "SENT: EHLO mail.mydomain.com"
"SMTPC" 5204 2102 "2014-08-26 12:54:45.950" "213.165.67.115" "RECEIVED: 250-gmx.net Hello mail.mydomain.com [111.222.333.444][nl]250-SIZE 157286400[nl]250 STARTTLS"
"SMTPC" 5204 2102 "2014-08-26 12:54:45.950" "213.165.67.115" "SENT: STARTTLS"
"SMTPC" 5168 2102 "2014-08-26 12:54:46.122" "213.165.67.115" "RECEIVED: 220 OK"
"TCPIP" 3016 "2014-08-26 12:54:57.073" "TCPConnection - SSL handshake with client failed. Error code: 335544539, Message: short read, Remote IP: 213.165.67.115"
"TCPIP" 5836 "2014-08-26 12:54:57.073" "Connecting to 213.165.67.115:25..."
"SMTPC" 11580 2103 "2014-08-26 12:54:57.198" "213.165.67.115" "RECEIVED: 220 gmx.net (mxgmx104) Nemesis ESMTP Service ready"
"SMTPC" 11580 2103 "2014-08-26 12:54:57.198" "213.165.67.115" "SENT: EHLO mail.mydomain.com"
"SMTPC" 11580 2103 "2014-08-26 12:54:57.244" "213.165.67.115" "RECEIVED: 250-gmx.net Hello mail.mydomain.com [111.222.333.444][nl]250-SIZE 157286400[nl]250 STARTTLS"
"SMTPC" 11580 2103 "2014-08-26 12:54:57.244" "213.165.67.115" "SENT: MAIL FROM:<user@mydomain.com>"
"SMTPC" 3204 2103 "2014-08-26 12:54:57.619" "213.165.67.115" "RECEIVED: 250 Requested mail action okay, completed"
"SMTPC" 3204 2103 "2014-08-26 12:54:57.619" "213.165.67.115" "SENT: RCPT TO:<argentumphoto@gmx.co.uk>"
"SMTPC" 7032 2103 "2014-08-26 12:54:57.806" "213.165.67.115" "RECEIVED: 250 OK"
"SMTPC" 7032 2103 "2014-08-26 12:54:57.806" "213.165.67.115" "SENT: DATA"
"SMTPC" 9748 2103 "2014-08-26 12:54:57.853" "213.165.67.115" "RECEIVED: 354 Start mail input; end with <CRLF>.<CRLF>"
"SMTPC" 9748 2103 "2014-08-26 12:54:57.853" "213.165.67.115" "SENT: [nl]."
"SMTPC" 7348 2103 "2014-08-26 12:54:57.993" "213.165.67.115" "RECEIVED: 250 Requested mail action okay, completed: id=0MfS8F-1WynTA0Gbb-00P994"
"SMTPC" 7348 2103 "2014-08-26 12:54:57.993" "213.165.67.115" "SENT: QUIT"
"SMTPC" 5168 2103 "2014-08-26 12:54:58.024" "213.165.67.115" "RECEIVED: 221 gmx.net Service closing transmission channel"
"APPLICATION" 5836 "2014-08-26 12:54:58.024" "SMTPDeliverer - Message 153: Message delivery thread completed."

User avatar
martin
Developer
Developer
Posts: 6846
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: STARTTLS feedback?

Post by martin » 2014-08-26 17:51

Yes, the '250 STARTTLS' response means that the server supports STARTTLS.

Is the issue reproducible for you?
Have you customized the set of allowed SSL/TLS ciphers?

Your own certificate is not used when delivering to external servers. Then their certificate is used, and whether or not you have a self-signed cert does not matter.

percepts
Senior user
Senior user
Posts: 5282
Joined: 2009-10-20 16:33
Location: Sceptred Isle

Re: STARTTLS feedback?

Post by percepts » 2014-08-26 17:53

This is very odd because now its working absolutely fine. I get received header at gmx as
Received: from mail.mydomain.com ([111.222.333.444]) by mx-ha.gmx.net (mxgmx011) with
ESMTPS (Nemesis) id 0MLBJ3-1XMZtz2p17-000H50 for <myuser@gmx.co.uk>;
So looks like gmx does receive via ESMTPS and same cert as before passed test this time.

So I'm thoroughly confused why one moment it doesn't work and next it does. Nothing changed at my end. Strange...

Our mails crossed...

No I'm not using ciphers.

percepts
Senior user
Senior user
Posts: 5282
Joined: 2009-10-20 16:33
Location: Sceptred Isle

Re: STARTTLS feedback?

Post by percepts » 2014-08-26 18:11

tried a few more send scenarios icluding restart of hmailserver and it's going through to gmx fine with succesfull cert verification using ESMTPS

I'll keep an eye on it and report back if it happens again. Although my gmx address is really only used for doing a few tests so its rarely used.

myname
New user
New user
Posts: 7
Joined: 2014-08-28 21:58

Re: STARTTLS feedback?

Post by myname » 2014-08-28 22:25

1) Do not use validation certificate trust for STARTTLS. At least not as a default option. You are probably the only one in the world who use for STARTTLS certificate validation requires. You are on very thin ice. In the current version is function STARTTLS in hMailServer with this condition rather useless.
2) You used the functionality validation certificate is surely incorrect. Indeed does not respect the alternative domain names. I would wrote you some mailservers (they have trusted certificates) with which it can try, but not so publicly to forum.

prisma
Senior user
Senior user
Posts: 325
Joined: 2010-07-09 13:16

Re: STARTTLS feedback?

Post by prisma » 2014-08-29 11:06

myname wrote: 1) Do not use validation certificate trust for STARTTLS. At least not as a default option. You are probably the only one in the world who use for STARTTLS certificate validation requires. You are on very thin ice. In the current version is function STARTTLS in hMailServer with this condition rather useless.
Only for Routes and Relay certificate validation should be active. From my point of view this behaviour is OK. Are you able to commit contrary observations?
myname wrote: 2) You used the functionality validation certificate is surely incorrect. Indeed does not respect the alternative domain names. I would wrote you some mailservers (they have trusted certificates) with which it can try, but not so publicly to forum.
@martin: As far as I can remember you do respect alternate names and wildcards, don't you? Is it currently implemented that way?

User avatar
martin
Developer
Developer
Posts: 6846
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: STARTTLS feedback?

Post by martin » 2014-08-29 18:43

Have not tested alternative names, but according to the documentation for the API:s I'm using it should work. I'll take a look at it. If you could give an example with a mail server / domain name where it does not work that would be helpful.

I also does not understand why the implementation would be on 'thin ice'. The certificate validation is done, but hMailServer proceeds regardless of result when a MX-resolved delivery is made. And for route / SMTP relayers it's optional if you want to require STARTTLS. So I don't see the problem.

User avatar
martin
Developer
Developer
Posts: 6846
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: STARTTLS feedback?

Post by martin » 2014-08-30 10:53

@myname: I just tried delivering to a TLS server with the matching host name in the Subject Alternative Names list rather than in the Common Names list and certificate verification succeeded as it should. Can you send me an example in a private message which it does not work for?

myname
New user
New user
Posts: 7
Joined: 2014-08-28 21:58

Re: STARTTLS feedback?

Post by myname » 2014-08-30 14:41

So you think that if the recipient mailserver certificate is untrusted to not encrypt the transmission is better? It's just that takle no one else in the world does not. If you accept the view that it is better to encrypt the transmission over unverified certificate then you not need to figure out how it works certificate verification. Drawing the lessons from how STARTTLS works in Microsoft Exchange, Kerio MailServer and IceWarp mailserver .....
In any case, thank you for dealing with my posts. Just trying to help, because your solution STARTTLS is not really applicable. I do not use hMailServer, but I know people who do. The STARTTLS feature for a long time looking forward now but are of your applications of this function sad.

If I try send PM "You cannot send private messages before you've made 5 posts in the user forum."

User avatar
martin
Developer
Developer
Posts: 6846
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: STARTTLS feedback?

Post by martin » 2014-08-30 15:48

myname wrote:So you think that if the recipient mailserver certificate is untrusted to not encrypt the transmission is better?
No I don't. Are you seeing this behavior with hMailServer or why do you think this is how it works?
Have you read the documentation about how it's supposed to work?
myname wrote:If I try send PM "You cannot send private messages before you've made 5 posts in the user forum."
Feel free to send it to me at martin at hmailserver.com. Or just make 3 dummy posts in "off topic" section.

You do realize that this is still a beta an open for feedback. No reason to get emotional and "sad" about the behavior. I recommend giving constructive feedback instead based on the documentation and the result you are seeing, because feeling "sad" will get you nowhere.

Post Reply