Certificate issue when sending and receiving email on 5.5

Use this forum if you want to discuss a problem or ask a question related to a hMailServer beta release.
Post Reply
rickyrick
New user
New user
Posts: 18
Joined: 2010-04-29 22:57

Certificate issue when sending and receiving email on 5.5

Post by rickyrick » 2014-09-27 23:06

I have just moved from 5.4 to 5.5 (before the 5.5 went back to beta), and when I am trying to send emails (via encrypted SMTP) and receive emails from external accounts (via encrypted POP) I am getting the following error :

Verify certificate: True, Expected remote host name: mail.btinternet.com
Certificate verification failed for session 6077. Expected host: mail.btinternet.com, Windows error code: -2146762480, Windows error message: The certificate is not valid for the requested usage."
TCPConnection - SSL handshake with client failed. Error code: 336134278, Message: certificate verify failed, Remote IP: 65.20.0.43"
I relay through BT.

Is there anyway to turn this certificate validation checking off ? I presume its new to 5.5 ?

I have had to revert back to non encryped sessions for now until I can resolve this.

Thanks

Richard

User avatar
mattg
Moderator
Moderator
Posts: 19877
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Certificate issue when sending and receiving email on 5.

Post by mattg » 2014-09-28 00:31

The certificate validation is new to 5.5

This should only be an issue because the certificate used by your ISP isn't setup correctly.
Contact your ISP.
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

rickyrick
New user
New user
Posts: 18
Joined: 2010-04-29 22:57

Re: Certificate issue when sending and receiving email on 5.

Post by rickyrick » 2014-09-28 00:40

mattg wrote:The certificate validation is new to 5.5

This should only be an issue because the certificate used by your ISP isn't setup correctly.
Contact your ISP.
Thanks for the quick reply.

Is there anyway to have an option in the ini file to disable this verification? I only ask as one of my email address hosting charge for a valid ssl cert for my email.

Richard

User avatar
mattg
Moderator
Moderator
Posts: 19877
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Certificate issue when sending and receiving email on 5.

Post by mattg » 2014-09-28 01:20

To be clear, you don't need an SSL cert
The SSL cert used by your ISP is incorrect.
They should fix it.

You are far better off using an unsecured connection, than using a secured connection to an invalid certificate.
At least then you don't expect your mail to be safe (Not that email is EVER safe unless you use message level encryption)
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
martin
Developer
Developer
Posts: 6831
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: Certificate issue when sending and receiving email on 5.

Post by martin » 2014-09-29 10:55

Could be that this is an issue in hMailServer. Will check the code.

prisma
Senior user
Senior user
Posts: 309
Joined: 2010-07-09 13:16

Re: Certificate issue when sending and receiving email on 5.

Post by prisma » 2014-09-29 11:01

I checked the certificate of this server. Certificate is valid and hostname fits. Can't see a reason for an error. Seems to be a bug. The server is resolved by a CNAME.

mattg wrote:You are far better off using an unsecured connection, than using a secured connection to an invalid certificate.
Sorry, but no. This is incorrect. But you're right the same time. It would be stupid to accept every certificate regardless the error occurring. For this reason I wrote at the very very beginning of the whole STARTTLS discussion: We need a certificate "pinning" function to accept selected certificates on the basis of fingerprints. E.g. every Mozilla Firefox or Thunderbird does it that way. It not my personal idea and requirement.

To accept invalid certificates also follows the SSL RFCs as long some conditions are fulfilled. E.g. the User/Admin has to be noticed (logfile/frontend), he has to be able to configure the behaviour and so on...

User avatar
martin
Developer
Developer
Posts: 6831
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: Certificate issue when sending and receiving email on 5.

Post by martin » 2014-09-29 17:26

I've reported this as an issue in hMailServer and committed a fix for 5.5.1.

Every certificate is issued with a specific purpose. Normally one of the purposes are "Ensures the identity of a remote computer" (also known as "1.3.6.1.5.5.7.3.1") and this is what hMailServer requires. But apparently there are servers out there who are using two additional obsolete purposes in their certificates (one of them being Netscape Server Gated Crypto), which mail.btinternet.com has in its chain. I've added these two obsolete purposes so that hMailServer allows them.

Chromium and Chrome seems to allow these two obsolete purposes, and according to a comment in the chromium source code IE does it as well, so it should be safe.

percepts
Senior user
Senior user
Posts: 5282
Joined: 2009-10-20 16:33
Location: Sceptred Isle

Re: Certificate issue when sending and receiving email on 5.

Post by percepts » 2014-10-01 03:22

someone has same problem on 5.5.1 B2097

viewtopic.php?f=7&t=27127&p=167298&sid= ... 0f#p167298

User avatar
mattg
Moderator
Moderator
Posts: 19877
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Certificate issue when sending and receiving email on 5.

Post by mattg » 2014-10-01 04:50

Just set up a second hmailserver at a location, with the sole job of connecting back to my main hmailserver via POP3 External download, and then dropping all attachments in a set directory. (Yes I know there was probably an easier way, but i did what I know would take less than 10 minutes to set up)

Couldn't connect via POP3 external download to my main hmailsevrer using StartTLS (required) or SSL/TLS, both ways my certificate was rejected.
My mail clients connect fine using this same certificate....

Have disabled all incoming StartTLS until I have time to investigate further...
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

prisma
Senior user
Senior user
Posts: 309
Joined: 2010-07-09 13:16

Re: Certificate issue when sending and receiving email on 5.

Post by prisma » 2014-10-01 11:01

If you inspect the certificate of the primary server on the secondary server, does Windows handle this certificate as valid?

Background of my question is: certificate validation is switched on for routes and relayer. In this case you set a server manually and it's expectable to get a valid certificate from this explicit server or to make an untrusted certificate trusted by windows certificate store. In case of POP3 it's the same. You configure an explicit server.

Maybe it's the same logic. And I think this would definitely make sense.
@Martin: what happens in case of POP3 fetching? Certificate validation?

User avatar
mattg
Moderator
Moderator
Posts: 19877
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Certificate issue when sending and receiving email on 5.

Post by mattg » 2014-10-01 14:44

how do I check that?

On the server doing the POP3 download I get this in the logs

Code: Select all

"DEBUG"	8588	"2014-10-01 10:09:13.699"	"Performing SSL/TLS handshake for session 7. Verify certificate: True, Expected remote host name: mail.mydomain.com.au"
"DEBUG"	8492	"2014-10-01 10:09:16.738"	"Certificate verification failed for session 7. Expected host: mail.mydomain.com.au, Windows error code: -2146762481, Windows error message: The certificate's CN name does not match the passed value."
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
martin
Developer
Developer
Posts: 6831
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: Certificate issue when sending and receiving email on 5.

Post by martin » 2014-10-01 16:00

Yes, SSL certificate verification takes place during POP3 fetching.

Here's one way to check the contents of the SSL cert. There's probably easier ways, but this is what I normally do:

1) Install OpenSSL from https://slproweb.com/products/Win32OpenSSL.html
2) Open a prompt and cd to C:\OpenSSL-Win32\bin
3) Execute openssl s_client -connect mail.hmailserver.com:465 (replace host name with yours)
4) Copy the server certificate part (section which starts with -----BEGIN CERTIFICATE----- and ends with -----END CERTIFICATE----- including these lines).
5) Save the certificate in file named test.crt
6) Double-click on the file test.crt in the Windows Explorer.

On the details tab, you should see "Subject" and "Subject Alternative Name" which should list the valid host names.

There appears to be Online tools to do this, such as https://www.sslshopper.com/ssl-checker.html. No idea how/whether these works though.

percepts
Senior user
Senior user
Posts: 5282
Joined: 2009-10-20 16:33
Location: Sceptred Isle

Re: Certificate issue when sending and receiving email on 5.

Post by percepts » 2014-10-01 16:33

goto start and type in mmc and run it.

In there fil/add snap-in certificates. Then save as certificates.msc which will put a shortcut in your administrative tools in your start menu.

in there you can view all of your installed certificates. Click on one to view its details.

And if your hMail cert is not in there then you can add it using action/all tasks/import and put it in one of your repositories such as trusted people/certificates.

You can edit your certs in details/select a detail and edit

prisma
Senior user
Senior user
Posts: 309
Joined: 2010-07-09 13:16

Re: Certificate issue when sending and receiving email on 5.

Post by prisma » 2014-10-01 20:18

.... And.... As Long as you don't access your primary Server by a hostname listet within the cert, possibly you want to access your server locally not via internet, you'll never get it working without further tricks.

In this case we use to manipulate DNS resolution by DNS Server or hosts file.

User avatar
mattg
Moderator
Moderator
Posts: 19877
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Certificate issue when sending and receiving email on 5.

Post by mattg » 2014-10-02 00:45

Well it also helps if I actually put the correct details in too.... :oops: :oops:

I recently created a new MX record for mx.mydomain.com.au and got a certificate for that.
I still have mail.mydomain.com.au as an MX record.

when I set up the POP3 external download I used mail.mydomain.com.au (in error)
If I change that to the correct mx.mydomain.com.au I get a different error on the machine doing the external POP3 download
"DEBUG" 9148 "2014-10-02 08:38:56.926" "Certificate verification failed for session 269. Expected host: mx.mydomain.com.au, Windows error code: -2146885614, Windows error message: The revocation function was unable to check revocation for the certificate."
"TCPIP" 9148 "2014-10-02 08:38:56.942" "TCPConnection - SSL handshake with client failed. Error code: 336134278, Message: certificate verify failed, Remote IP: MY.IP.WAS.HERE"
Matt
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

prisma
Senior user
Senior user
Posts: 309
Joined: 2010-07-09 13:16

Re: Certificate issue when sending and receiving email on 5.

Post by prisma » 2014-10-02 10:29

Now the hostname seems to fit, but CRL check was not possible. Please inspect the CRL distribution points and OCSP responder listed within the cert. Are they accessible? Doesn't look like that...

(I know I'm getting on everybody's nerves. But I'll say it anyway: Pinning of certs along with allowed errors would help in such situations)

User avatar
mattg
Moderator
Moderator
Posts: 19877
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Certificate issue when sending and receiving email on 5.

Post by mattg » 2014-10-02 17:49

OK so how / where do I find these things?
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

prisma
Senior user
Senior user
Posts: 309
Joined: 2010-07-09 13:16

Re: Certificate issue when sending and receiving email on 5.

Post by prisma » 2014-10-02 21:32

As Martin said. Copy the certificate part of the output of openssl s_client to a .cer file and double click it. Or just double click the certificate on the primary server. Within the certificate details you should be able to find the info...

User avatar
mattg
Moderator
Moderator
Posts: 19877
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Certificate issue when sending and receiving email on 5.

Post by mattg » 2014-10-02 23:45

OK I can see a CRL Distribution Point, which looks OK and contains a valid URL

I can't see OCSP at all.

I've trialled the certificate on it's own, with Root CA details both above and below the certificate as per ideas here
viewtopic.php?f=12&t=22371

It seems to work fine with mail clients, I've not seen any issues with connections with other servers, but then I'm noticing heaps of stuff not showing up in logs either during my testing of this - so maybe I'm missing a lot of mail overall...
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
mattg
Moderator
Moderator
Posts: 19877
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Certificate issue when sending and receiving email on 5.

Post by mattg » 2014-10-03 00:07

This line is the only one that says that the connection didn't work at the main server.
"DEBUG" 42736 "2014-10-03 08:02:55.566" "The read operation failed. Bytes transferred: 0 Remote IP: XXX.XXX.XXX.XXX, Session: 13, Code: 2, Message: End of file"
I get a very similar line whenever someone triggers greylisting.

I really don't know how much mail I'm missing.
I have removed StartTLS from port 25 for the time being.
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
mattg
Moderator
Moderator
Posts: 19877
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Certificate issue when sending and receiving email on 5.

Post by mattg » 2014-10-03 00:10

Another thought, can the hMailserver diagnostics be set to somehow help diagnose / test certificates used on the server
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

prisma
Senior user
Senior user
Posts: 309
Joined: 2010-07-09 13:16

Re: Certificate issue when sending and receiving email on 5.

Post by prisma » 2014-10-03 21:03

Good idea. But your problem sounds like Martin should do some research. I think you'd better send him the cert or url to your server and let him check the problem. It's important to get 5.5 stable.

User avatar
martin
Developer
Developer
Posts: 6831
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: Certificate issue when sending and receiving email on 5.

Post by martin » 2014-10-03 21:31

mattg, can you zip and send the certificate to me at martin@hmailserver.com?

If you reproduce the issue again, do you still see the same "The revocation function was unable to check revocation for the certificate."?
Martin Knafve
martin@hmailserver.com
https://twitter.com/knafve

User avatar
mattg
Moderator
Moderator
Posts: 19877
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Certificate issue when sending and receiving email on 5.

Post by mattg » 2014-10-04 01:01

cert sent...

entirely reproducible
sent you details by email as well
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
martin
Developer
Developer
Posts: 6831
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: Certificate issue when sending and receiving email on 5.

Post by martin » 2014-10-04 10:17

I just set up my local hMailServer (latest 5.5.1) to do a POP3 fetch from the host name you gave me and the certificate verification was succesful:
"DEBUG" 3196 "2014-10-04 09:45:12.707" "Performing SSL/TLS handshake for session 7. Verify certificate: True, Expected remote host name: yourhost.example.com"
"DEBUG" 3196 "2014-10-04 09:45:15.838" "Certificate verification succeeded for session 7."
Just some background so you get the picture. I don't know how much of this you already know, so ... hMailServer uses the Windows Crypto API to verify certifictes. As part of this verification Windows Crypto API checks if the certificate has been revoked. A certificate should be revoked if a hacker has gained access to it.

The revocation check could fail for a number of reasons, such as:
  • The site hosting the revocation list (by the certificate authority) is offline. The list is cached locally, but if it's unavailable when needed then that would cause a failure.
  • The process (hMailServer.exe in this case) trying to do the certificate verification for some reason unable to access the revocation list. This could be caused if there's some kind of proxy or similar which http traffic needs to go through, but hMailServer does not do that.
  • There's a timeout when trying to download the certificate list. I believe the Windows timeout by default is 15 seconds.
Things you could try:

1) To determine if it's a timeout issue, check how long time the certificate verification takes. You would do that by comparing the timestamps of "Performing SSL/TLS handshake for session 7" and "Certificate verification succeeded for session 7" in the log. I don't think this is the issue though, becuase as I remember it Windows report a specific error message if there's a timeout. If it's a timeout (time is similar to 15 seconds), you could try extending the timeout. I've never tried this: Start gpedit.msc. Select Computer Configuration -> Windows Settings -> Security settings -> Public Key Policies. Select Certificate Path Validation Settings in the list to the right. Select the "Network Retrieval" tab and increase the default timeouts there. Unfortunately no idea whether a reboot is neccessary.
2) Maybe try running hMailServer with your own Windows user account rather than Local System to see if that changes things. If you've set up a proxy in Windows that may be on a per-user basis.
3) Confirm that you're not using any kind of proxy/anti virus/etc which could block http traffic from the hMailServer service. If you're running something like that, maybe try to temporarily stop it.
4) Open up a command prompt and run certutil -verify C:\path\to\cert.crt. This will do a certificate verification using the same API as hMailServer. It would be interesting to see if you get the same error here.
Martin Knafve
martin@hmailserver.com
https://twitter.com/knafve

rickyrick
New user
New user
Posts: 18
Joined: 2010-04-29 22:57

Re: Certificate issue when sending and receiving email on 5.

Post by rickyrick » 2014-10-05 11:34

Hi

I've installed the latest build 2103, and the btinernet.com cert is now working. Thanks :-)

"Certificate verification succeeded for session 8."

Richard

User avatar
mattg
Moderator
Moderator
Posts: 19877
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Certificate issue when sending and receiving email on 5.

Post by mattg » 2014-10-05 13:12

martin wrote:Things you could try:

3) Confirm that you're not using any kind of proxy/anti virus/etc which could block http traffic from the hMailServer service. If you're running something like that, maybe try to temporarily stop it.
Thanks for the hints and help.

AVAST on the remote hMailserver machine was blocking this.

I didn't pick that because I also have AVAST running on my main server, and don't have to stop it checking mail.

Thanks
*** All Sorted
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
martin
Developer
Developer
Posts: 6831
Joined: 2003-11-21 01:09
Location: Sweden
Contact:

Re: Certificate issue when sending and receiving email on 5.

Post by martin » 2014-10-05 13:18

Ah, fun stuff. Was it some per-process configuration? Guess it could be a bad idea to disable it completely.
Martin Knafve
martin@hmailserver.com
https://twitter.com/knafve

User avatar
mattg
Moderator
Moderator
Posts: 19877
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Certificate issue when sending and receiving email on 5.

Post by mattg » 2014-10-05 15:14

For the time being I have just stopped it from checking mail traffic...
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

User avatar
mattg
Moderator
Moderator
Posts: 19877
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Certificate issue when sending and receiving email on 5.5

Post by mattg » 2016-12-25 01:14

Just moving server again and got this issue again
Fix was to go to Avast mail shield settings and disallow SSL scanning (AV creates a man in the middle type SSL cert to inspect mail - mail is later sent to Avast via command line anyway - and to ClamAV and SpamAssassin then clamAV again)
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

Post Reply