SSL handshake failed; sslv3 alert certificate unknown

Use this forum if you have installed hMailServer and want to ask a question related to a production release of hMailServer. Before posting, please read the troubleshooting guide. A large part of all reported issues are already described in detail here.
Post Reply
Elte156
New user
New user
Posts: 3
Joined: 2017-02-10 18:51

SSL handshake failed; sslv3 alert certificate unknown

Post by Elte156 » 2017-02-10 20:45

I'm posting this for future reference to help others that could experience a similar problem. (I've already searched the forums and only found 2 posts that did not help)
I've read the hMail SSL documentation too and it was not enough to find the solution.
Problem: Connecting to hMail server through an IMAP SSL (IMAPS) connection on port 993

hMail verison: 5.6.5-B2367

SSL Certificate Settings (Signed by a trusted CA. These are not self-signed!)
Certificate File: C:\SSL\web\STAR_domain_com.crt
Private Key File: C:\SSL\web\STAR_domain_com.key
SSL/TLS
Verify remote server SSL/TLS certificates: TRUE
Versions: TLS v1.0, TLS v1.1, TLS v1.2
TCP/IP Port Settings
Protocol: IMAP
IP: 0.0.0.0
Port: 993
Security: SSL/TLS
Certificate: STAR_domain_com

Trying to connect via from a custom JAVA mail client:
hMail Logs
"DEBUG" 5880 "2017-02-10 09:49:29.378" "Creating session 3402"
"TCPIP" 5880 "2017-02-10 09:49:29.379" "TCP - X.X.X.X connected to X.X.X.X:993."
"DEBUG" 5880 "2017-02-10 09:49:29.395" "TCP connection started for session 2660"
"DEBUG" 5880 "2017-02-10 09:49:29.396" "Performing SSL/TLS handshake for session 2660. Verify certificate: False"
"TCPIP" 5872 "2017-02-10 09:49:29.619" "TCPConnection - TLS/SSL handshake failed. Session Id: 2660, Remote IP: X.X.X.X, Error code: 336151574, Message: sslv3 alert certificate unknown"
"DEBUG" 5872 "2017-02-10 09:49:29.620" "Ending session 2660"
Mail Client Logs:
sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.validator.PKIXValidator.doBuild(Unknown Source)
at sun.security.validator.PKIXValidator.engineValidate(Unknown Source)
at sun.security.validator.Validator.validate(Unknown Source)
at sun.security.ssl.X509TrustManagerImpl.validate(Unknown Source)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(Unknown Source)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(Unknown Source)
at sun.security.ssl.ClientHandshaker.serverCertificate(Unknown Source)
at sun.security.ssl.ClientHandshaker.processMessage(Unknown Source)
at sun.security.ssl.Handshaker.processLoop(Unknown Source)
at sun.security.ssl.Handshaker.process_record(Unknown Source)
at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)
at sun.security.ssl.SSLSocketImpl.writeRecord(Unknown Source)
at sun.security.ssl.AppOutputStream.write(Unknown Source)
at sun.security.ssl.AppOutputStream.write(Unknown Source)
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.provider.certpath.SunCertPathBuilder.build(Unknown Source)
at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(Unknown Source)
at java.security.cert.CertPathBuilder.build(Unknown Source)
... 16 more
This is what I started with and I struggled for the longest time. But I figured out the simple, but obscure solution.

Solution: Make sure your public certificate (hMail Certificate File settings) contains your entire trusted chain!
My trusted CA provided 2 files when I exported my cert. I got the public CRT and the CA-BUNDLE files.
I noticed my apache server needed 3 items: public cert, private key, ca-bundle.
hMail only requests 2 items: public cert, private key.
After I combined the public cert and ca-bundle into 1 file, hMail no longer had handshake failures!

Cheers. :D

User avatar
RvdH
Senior user
Senior user
Posts: 1506
Joined: 2008-06-27 14:42
Location: Netherlands

Re: SSL handshake failed; sslv3 alert certificate unknown

Post by RvdH » 2017-02-10 22:34

I really have my doubts about your conclusions.
I only see these 'sslv3 alert certificate unknown' errors in my logs if someone is trying to use SSLv3 (which s not enabled on my server)

As far i can see above you mentioned you only enabled: TLS v1.0, TLS v1.1, TLS v1.2 and thus NOT SSLv3 connections what would explain the 'sslv3 alert certificate unknown' messages
CIDR to RegEx: d-fault.nl/CIDRtoRegEx
DNS Lookup: d-fault.nl/DNSTools
DNSBL Lookup: d-fault.nl/DNSBLLookup
GEOIP Lookup: d-fault.nl/GeoipLookup

User avatar
mattg
Moderator
Moderator
Posts: 21533
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: SSL handshake failed; sslv3 alert certificate unknown

Post by mattg » 2017-02-10 23:04

Elte156 wrote:I've read the hMail SSL documentation too and it was not enough to find the solution.
Please feel free to post some suggested changes and I will look at them...

I am currently getting this message from ONE mail client at ONE location, and I am trying to track down which mail client it is. Every time I think I have it, I realise later that I am wrong as this error still happens.
I do have SSLv3 enabled, and I do have chained certificates. My connection is POP3 over SSL on port 995.

Chaining certificates has long been required for some mail clients to accept the certificate.

I do agree with RvdH though in this instance
RvdH wrote:As far i can see above you mentioned you only enabled: TLS v1.0, TLS v1.1, TLS v1.2 and thus NOT SSLv3 connections what would explain the 'sslv3 alert certificate unknown' messages
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

Elte156
New user
New user
Posts: 3
Joined: 2017-02-10 18:51

Re: SSL handshake failed; sslv3 alert certificate unknown

Post by Elte156 » 2017-02-10 23:22

Let me postface my initial post with;
I've tried almost all permutations regarding the hMail SSL settings via the Windows GUI.

Includes:
Opened up ALL securities (TLS1.0+ and SSLv3), tried to connect... same error
Unchecked SSL verification, tried to connect... same error
Changed port 993 to be NONE or STARTTLS instead of SSL/TLS, tried to connect... different errors like "Unrecognized SSL message, plaintext connection?;" etc...


Thinking the problem was occurring in my custom mail client. So I removed that from testing;
Used openssl cmds to connect to my apache server on :443. No problem.
Used openssl to connect to hmail :993, I get that SSLv3 error.
But my apache server and hMail use the same exact cert. :roll:


Took a look at the results from openssl.
Apache supplied 1, 2, 3 certs (public + ca-bundle)
I see "Verify return code: 0 (ok)"

hMail supplied 1 cert (public)
I see "Verify return code: 21 (unable to verify the first certificate)"
Seems to point that the entire chain is not being included. :roll:


This error can easily be reproduced if I just remove the intermediate and root certs (ca-bundle) from my referenced public cert and restart hMail.
I'm sure it's some uncaught exception that is happening in hMail that bubbles up during the selection of an agreed protocol (ssl3, tls1.0, etc..)
It's returning "SSLv3" in the error; well because that's probably the first protocol it tries to make an agreement with...

User avatar
mattg
Moderator
Moderator
Posts: 21533
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: SSL handshake failed; sslv3 alert certificate unknown

Post by mattg » 2017-02-10 23:42

Elte156 wrote:hMail supplied 1 cert (public)
I see "Verify return code: 21 (unable to verify the first certificate)"
Seems to point that the entire chain is not being included. :roll:
mattg wrote:Chaining certificates has long been required for some mail clients to accept the certificate.
In fact this was introduced and noted first in ver 5.4.
Prior to that we weren't able to chain certs.

Chaining is NOT required for most mail clients (they simply don't care), and is redundant if your AntiVirus inspects SSL connections, AV in these cases deliberately make a man-in-the-middle scenario.

Elte156 wrote:This error can easily be reproduced if I just remove the intermediate and root certs (ca-bundle) from my referenced public cert and restart hMail.
For that particular mail client that may be correct.

I have dozens of mail clients that connect without issue, I have certs chained, and yet I get this error from one as yet unknown mail client from one location, and only recently.
Elte156 wrote:I'm sure it's some uncaught exception that is happening in hMail that bubbles up during the selection of an agreed protocol (ssl3, tls1.0, etc..)
It's returning "SSLv3" in the error; well because that's probably the first protocol it tries to make an agreement with...
That may be correct. I'll have a bit of a look around the code and see if I can find it...

Technically speaking though, shouldn't mail clients start at the top level of security and work down the list, not start at the bottom and work up...
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

Elte156
New user
New user
Posts: 3
Joined: 2017-02-10 18:51

Re: SSL handshake failed; sslv3 alert certificate unknown

Post by Elte156 » 2017-02-11 00:48

mattg wrote:Technically speaking though, shouldn't mail clients start at the top level of security and work down the list, not start at the bottom and work up...
My mail client is doing just that. But by default, the java mail client actually disables the use of SSLv3. Here's what happens if I ONLY enable SSLv3 on hMail.
javax.mail.MessagingException: Server chose SSLv3, but that protocol version is not enabled or not supported by the client.;
This entire time during the tests with my client, it always tried to connect with TLS1.0 and never SSLv3.
So what would cause hMail to complain about "sslv3 alert certificate unknown"?
In order to use TLS1.1, 1.2 I have to explicitly tell the client to use those protocols and if I did, it will follow the "top to bottom" security you just mentioned.

Anti-virus is not an issue at this point, because I don't have it installed on my dev env.

So regardless if it's my custom email client or a popular one like thunderbird; hMail is throwing out false errors about SSLv3 when it has to do with public certs and their trusted chains. I would of been save a lot of time if it said something that OpenSSL reported:
hMail supplied 1 cert (public)
I see "Verify return code: 21 (unable to verify the first certificate)
"

User avatar
mattg
Moderator
Moderator
Posts: 21533
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: SSL handshake failed; sslv3 alert certificate unknown

Post by mattg » 2017-02-11 01:13

Agreed, better error reporting is required.

I'll look into the code >> https://github.com/hmailserver/hmailserver
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

andreyzh
New user
New user
Posts: 2
Joined: 2017-10-17 13:41

Re: SSL handshake failed; sslv3 alert certificate unknown

Post by andreyzh » 2017-10-17 14:07

Good day,

I've been stumbling upon the same problem with the hMailServer. Upon trying to establish STARTTLS connection with my server on port 587 I get

Code: Select all

Error code: 336151574, Message: sslv3 alert certificate unknown
Configuration:
SMTP on 25, STARTTLS (Optional) -> Works OK
IMAP on 143, STARTTLS (Required) -> Works OK
SMTP on 587, STARTTLS (Optional: OK) (Required: Problem)

As for the certificate: I'm using the certificate from Let's encrypt by combining the certificate and issuer .pems in one file (please excuse sloppy description).
It works well with the IMAP, but for some reason clients trying to submit the message to the server refuse refuse to accept it. Tried with Jenkins Emailer and standard Android client.

Any ideas where the problem might be?
I understand that might not be sufficient information to help out with this, so please let me know what else could be useful here?

User avatar
mattg
Moderator
Moderator
Posts: 21533
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: SSL handshake failed; sslv3 alert certificate unknown

Post by mattg » 2017-10-17 14:12

Lets Encrypt have a number of certificate files

use the fullchain.pem as the certificate, and the privatekey.pem as the key

There is no need to combine certificates manually.

Looks to me as though the mail client wants to use SSLv3.0 which is likely unchecked in Advanced >> SSL (It should be unchecked as it is very insecure)
Which mail client are you using?
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

andreyzh
New user
New user
Posts: 2
Joined: 2017-10-17 13:41

Re: SSL handshake failed; sslv3 alert certificate unknown

Post by andreyzh » 2017-10-17 22:17

Thanks! Using fullchain helped with Android email client, and Jenkins just had to be configured to use TLS, since SSL was disabled on hMailServer (facepalm). Problem solved :)

User avatar
mattg
Moderator
Moderator
Posts: 21533
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: SSL handshake failed; sslv3 alert certificate unknown

Post by mattg » 2017-10-17 23:50

Thanks for the post back - glad you have it sorted
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

Post Reply