Using OpenSSL, trying to get a working SSL certificate

Use this forum if you want to discuss a problem or ask a question related to a hMailServer beta release.
Post Reply
telicatessin
New user
New user
Posts: 5
Joined: 2015-07-04 16:51

Using OpenSSL, trying to get a working SSL certificate

Post by telicatessin » 2015-07-04 17:31

Hey All, I first want to say I really enjoy hmail so far, thank you to the developers for creating a great mail application.

I work at a small television station and I am trying to deploy hmail on a windows server 2008 r2. We used to use citadel on an ubuntu system but it kind of took a dump. I currently have it installed with MySQL database on a 2TB Dell Poweredge Server. I really like this setup, so far i have been able to create 60 plus users on a laptop and transfer them over to this server through MySQL. It seems to be running smooth. However. I have been trying to set up an official SSL certificate and Im not having any luck. So Far I have done the following.

*Successfully generated a request .csr file and a private key .key file using OpenSSL
* copied the .key file over to a directory where it could be read from hmail later, and copied the contents of the .csr file over to a csr online request form for companies such as comodo, thawte, and now ssl.com

*When I received an email back, It contained the intermediate.crt, Root.crt, and the certificate for my domain.crt, I combined these into one .crt file like so from top to bottom..
-----BEGIN CERTIFICATE-----
<my_domain_com.crt>
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
<Intermediate.crt>
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
<RootCA.crt>
-----END CERTIFICATE-----
now I hade just the one domain.crt file with these combined and my private key from when i generated the private key .key.

then just to make sure (I had a key mismatch error when I tried this with comodo, At this point i was using the certificates from thawte which they onley give you 1 your_domain_com.crt, 1 Intermediate.crt, and 1 RootCA.crt which made things easy) I used this command in openssl just to check to see if the modules matched up so I didn't have a key mismatch error again..

* ( to check the modulus of the certificate that I just combined with the intermediate and the root above)
openssl x509 -noout -modulus -in my_domain_com.crt | openssl md5

* (to check the modulus of the private key that was generated initially)
openssl rsa -noout -modulus -in my_domain_com.key | openssl md5

And they matched, So I knew that I should not receive a key mismatch error. I went into hmail, deleted the error logs and loaded up my new my_domain_com.crt and my private.key into the SSL/TLS menu. I then checked ports 495 and ports 995 and made sure my certificate was selected and set for the correct protocol and set it them to listen on 0.0.0.0 (all addresses) Then restarted my server completely.
when it booted back up i did not have any errors, so it appeared that it loaded my ssl and private key correctly.

However, I loaded up thunderbird on the same machine (localhost) and tried to sign into an account using ssl/tls and Thunderbird could not find the settings for my account. It did work when I wasn't using SSL, using the default ports. I thought that maybe I could not connect on the local host because the ports were being used by hmail. So I tried it on my laptop on the same network. Still, no avail. At this point, I noticed an error log was created in hmail. The following error had occurred.

"Performing SSL/TLS handshake for session 33. Verify certificate: False"
"TCPIP" 4012 "2015-07-03 21:02:19.316" "TCPConnection - TLS/SSL handshake completed. Session Id: 33, Remote IP: 192.168.0.202, Version: TLSv1.2, Cipher: ECDHE-RSA-AES128-GCM-SHA256, Bits: 128"
"POP3D" 4012 33 "2015-07-03 21:02:19.316" "192.168.0.202" "SENT: +OK POP3"
"DEBUG" 3996 "2015-07-03 21:02:19.317" "The read operation failed. Bytes transferred: 0 Remote IP: 192.168.0.202, Session: 33, Code: 336151576, Message: tlsv1 alert unknown ca"
"DEBUG" 4012 "2015-07-03 21:02:19.317" "Ending session 33"

I thought this was odd because I did all of the necessary steps to get this to work and it only throws this error when I try to connect to it. I thought that maybe there was something wrong with my certificate from thawte. It is a trial and when I open it with windows shell it lists as being given to me from thawte but it shows this..
Untitled.png
Untitled.png (9.75 KiB) Viewed 5369 times
Not being verified by windows or unable to verify.

When I open the certificate settings for the server in thunderbird, thunderbird is unable to verify and wants me to add a certificate exception. I believe with a rootCA in my my_domain_com.crt I should not have to do that because they are linked.

I dont't know what I'm supposed to do with hmail/Externals/CA folder, that is not located in my hmail directory, I am running hmail version 5.6.3-B2249,

all i have is the .crt and the .key files selected in hmail as well as the ports. I left the password blank during the key generation, it did not ask me for a password when opening the .crs in notpad ++ and when i ran the command

openssl rsa -in your_certificatedomain_com.key -out your_certificatedomain_com.key

in open ssl and it changed the module for my comodo private key, so it created a mismatch error.

What should I do???

telicatessin
New user
New user
Posts: 5
Joined: 2015-07-04 16:51

Re: Using OpenSSL, trying to get a working SSL certificate

Post by telicatessin » 2015-07-04 21:07

So i ended creating a 90 day ssl certificate at SSL.com and combining the certificates together in one final .crt. This time, windows recognizes the Certificate but now I am getting this error when I try to connect using SSL in thunderbird.

"DEBUG" 4716 "2015-07-04 14:49:47.488" "Creating session 17"
"TCPIP" 4716 "2015-07-04 14:49:47.488" "TCP - 127.0.0.1 connected to 127.0.0.1:995."
"DEBUG" 4716 "2015-07-04 14:49:47.488" "TCP connection started for session 16"
"DEBUG" 4716 "2015-07-04 14:49:47.504" "Performing SSL/TLS handshake for session 16. Verify certificate: False"
"TCPIP" 4120 "2015-07-04 14:49:47.535" "TCPConnection - TLS/SSL handshake completed. Session Id: 16, Remote IP: 127.0.0.1, Version: TLSv1.2, Cipher: ECDHE-RSA-AES128-GCM-SHA256, Bits: 128"
"POP3D" 4120 16 "2015-07-04 14:49:47.535" "127.0.0.1" "SENT: +OK POP3"
"DEBUG" 4120 "2015-07-04 14:49:47.550" "The read operation failed. Bytes transferred: 0 Remote IP: 127.0.0.1, Session: 16, Code: 336151570, Message: sslv3 alert bad certificate"
"DEBUG" 4120 "2015-07-04 14:49:47.550" "Ending session 16"

User avatar
jimimaseye
Moderator
Moderator
Posts: 8859
Joined: 2011-09-08 17:48

Re: Using OpenSSL, trying to get a working SSL certificate

Post by jimimaseye » 2015-07-04 21:21

You read this? viewtopic.php?f=21&t=28255 There might be something in there that you have missed but is relevant.
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

telicatessin
New user
New user
Posts: 5
Joined: 2015-07-04 16:51

Re: Using OpenSSL, trying to get a working SSL certificate

Post by telicatessin » 2015-07-04 21:45

Thanks but maybe your can help me better understand what im supposed to do exactly, I've been working on this for some time and my brain cells are starting to fry.

I have recieved an email from SSL.com with a zip file with the following..

* Root CA Certificate - AddTrustExternalCARoot.crt
* Intermediate CA Certificate - USERTrustRSAAddTrustCA.crt
* Intermediate CA Certificate - SSLcomDVCA_2.crt
* Your Certificate (X.509) - my_domain_com.crt

most tutorials mention 1 Intermediate certificate, but I have two and it's very confusing. According to this post viewtopic.php?f=21&t=28255
I have resolved this myself, It is way simplier than what you read online!

If you have received a regular cert and an intermediate cert all you have to do is open the intermediate cert in notepad and open the root cert in notepad and copy the intermediate cert under the root certs
does this mean I open the Root CA AddTrustExternalCARoot.crt and copy and paste from the two intermediate certificates so it reads...
-----BEGIN CERTIFICATE-----
<AddTrusExternalCARoot.crt>
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
<USERTrustRSAAddTrustCA.crt>
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
<SSLcomDVCA_2.crt>
-----END CERTIFICATE-----
so that the root cert contains both and then save it...
So then I guess the name of the new combined .crt file does not matter??

Next it says
Then copy the new root cert to C:\Program Files (x86)\hMailServer\Externals\CA and the .key file you should have already created using openssl. Open up Hmail Administrator and go to Advanced->SSL certificates and add a new cert. Name is irrelevant but I read to give it the same common name as your CSR i.e. mail.your_dns_name.com. point the certificate file option to your cert (.crt) you copied over and the Private Key file (.key) you copied over.
However, My Hmail does not have "C:\Program Files (x86)\hMailServer\Externals\CA" I would have to create the directory \Externals\CA myself.
My hmail server is installed on a 2TB drive labled E:\Mail\hmailserver I would have to create the directory \Externals\CA myself.

Then go into Hmail and into SSL/TLS settings and choose my .crt file in E:\Mail\hmailserver\Externals\CA and my private key .key file in E:\Mail\hmailserver\Externals\CA

As far as my Your Certificate (X.509) - my_domain_com.crt, I don't do anything with this??

Just to clarify to make sure I am doing this correctly, thanks for your response.

telicatessin
New user
New user
Posts: 5
Joined: 2015-07-04 16:51

Re: Using OpenSSL, trying to get a working SSL certificate

Post by telicatessin » 2015-07-04 21:49

Also, for the time being, my hmail is not configured with the same domain in the certificate, i have it set to test1.mydomain.com just for internal network testing purposes, my certificate is for my.domain.com, can this have this effect?

telicatessin
New user
New user
Posts: 5
Joined: 2015-07-04 16:51

Re: Using OpenSSL, trying to get a working SSL certificate

Post by telicatessin » 2015-07-04 22:09

I Did the above and I get

"ERROR" 5020 "2015-07-04 16:08:12.775" "Severity: 2 (High), Code: HM5113, Source: SslContextInitializer::InitServer, Description: Failed to load private key file. Path: E:\WENY_Mail\hmailserver\Externals\CA\mail_mydomain_com.key, Address: 0.0.0.0, Port: 995, Error: use_private_key_file: key values mismatch

User avatar
mattg
Moderator
Moderator
Posts: 21198
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Using OpenSSL, trying to get a working SSL certificate

Post by mattg » 2015-07-05 03:36

telicatessin wrote:most tutorials mention 1 Intermediate certificate, but I have two and it's very confusing. According to this post viewtopic.php?f=21&t=28255
I have resolved this myself, It is way simplier than what you read online!

If you have received a regular cert and an intermediate cert all you have to do is open the intermediate cert in notepad and open the root cert in notepad and copy the intermediate cert under the root certs
does this mean I open the Root CA AddTrustExternalCARoot.crt and copy and paste from the two intermediate certificates so it reads...
-----BEGIN CERTIFICATE-----
<AddTrusExternalCARoot.crt>
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
<USERTrustRSAAddTrustCA.crt>
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
<SSLcomDVCA_2.crt>
-----END CERTIFICATE-----
so that the root cert contains both and then save it...
So then I guess the name of the new combined .crt file does not matter??
correct, except that you need to ALSO add your personal certificate to the end of this single file
telicatessin wrote: Next it says
Then copy the new root cert to C:\Program Files (x86)\hMailServer\Externals\CA and the .key file you should have already created using openssl. Open up Hmail Administrator and go to Advanced->SSL certificates and add a new cert. Name is irrelevant but I read to give it the same common name as your CSR i.e. mail.your_dns_name.com. point the certificate file option to your cert (.crt) you copied over and the Private Key file (.key) you copied over.
However, My Hmail does not have "C:\Program Files (x86)\hMailServer\Externals\CA" I would have to create the directory \Externals\CA myself.
My hmail server is installed on a 2TB drive labled E:\Mail\hmailserver I would have to create the directory \Externals\CA myself.
Depends on the version of hMailserver that you are using. This changed recently, and I expect that that the tutorial that you followed is for an older version.
The external\CA folder is to store details of other servers root certs, not your certificate details
telicatessin wrote:Also, for the time being, my hmail is not configured with the same domain in the certificate, i have it set to test1.mydomain.com just for internal network testing purposes, my certificate is for my.domain.com, can this have this effect?
yes it should generate an additional window in your mail client asking if you wish to accept the certificate
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

henrix
New user
New user
Posts: 4
Joined: 2021-01-08 16:58

Re: Using OpenSSL, trying to get a working SSL certificate

Post by henrix » 2021-01-09 16:34

Hi

that's unbelievable. I created sign self certificate and Ios Outlook now working but thunderbird not.

When thunderbird is connect then hmailserver return this error: TcpConnection - Tls/SSl handshake failed.Session Id:X Remote IP:x.x.x.x,Error code: 335544539, Message:short read.


but outlook is working and that is primary goal.

Thank you

User avatar
mattg
Moderator
Moderator
Posts: 21198
Joined: 2007-06-14 05:12
Location: 'The Outback' Australia

Re: Using OpenSSL, trying to get a working SSL certificate

Post by mattg » 2021-01-10 03:05

look for a background 'window' in thunderbird, that allows you accept the self signed certificate
Just 'cause I link to a page and say little else doesn't mean I am not being nice.
https://www.hmailserver.com/documentation

Post Reply