Lego: Windows-based auto-renewal of Let's Encrypt certs, including wildcards

Use this forum if you have installed hMailServer and want to ask a question related to a production release of hMailServer. Before posting, please read the troubleshooting guide. A large part of all reported issues are already described in detail here.
Post Reply
User avatar
ras07
Normal user
Normal user
Posts: 228
Joined: 2010-03-11 08:51

Lego: Windows-based auto-renewal of Let's Encrypt certs, including wildcards

Post by ras07 » 2019-04-17 10:05

I've been using Let's Encrypt certs for several years now. Their certs are only good for 90 days, and while there are some well-known ways to auto-renew them on Linux (which is what I've been doing, and then copying them over to my hMailServer machine) Windows support is pretty hit-or-miss. I wanted a system that:
  • has a completely automated, never-have-to-think-about-it renewal process
  • is a pure-Windows solution
  • is one that doesn't rely on installing a web server, Linux shell emulator, or other bulky system on my hMailServer box
  • Supports wildcard certs
After some research and a fair bit of trial and error, I have a solution I'm pretty happy with. The process isn't too involved but it is pretty sparsely documented, so I thought I'd write it up here.

(This ended up way longer than I expected, but it's really not hard ... and a bunch of this only applies if you run your own DNS server.)
  1. Download the Windows build of the Lego client. Lego is a Let's Encrypt client written is Go, and it seems to be pretty stable and under active maintenance - more than you can say for a lot of the Windows clients out there. (I won't call it well-documented, though.) The repository is at https://github.com/go-acme/lego, and the downloadable releases are at https://github.com/go-acme/lego/releases. Download the *_windows_386.zip or *_windows_amd64.zip file, as appropriate. As of this writing, v2.4.0 is current. (Chrome gave me a "this file is not commonly downloaded and may be dangerous" warning on the 32-bit version; the 64-bit file gave me no such warning. I scanned both files and didn't find malware in either. Chrome can be like that sometimes; I think it just wants you to feel bad about still being on a 32-bit OS.) The program is a single standalone EXE with no install required. I made a new directory, hmailserver/letsencrypt, and stuck everything in there.
  2. Wildcard certs require DNS-based validation rather than having you create a special file for download from your web server. This was fine with me because I didn't want to install a web server on my hmailserver box anyway. (DNS validation works for non-wildcard certs also, of course.) This works by creating a special DNS TXT record for the domain(s) you want the cert for. Lego can automatically add the appropriate TXT records for 50+ popular DNS providers, a list of which you can find at https://go-acme.github.io/lego/dns/azure/. Most of them utilize some kind of token or access key. There's no documentation in Lego about how to find or generate that token, so consult your DNS provider's docs.
  3. If you have your own DNS server you can use RFC2136 TSIG authentication to update DNS records, which is what I did. (Some of the supported DNS providers use TSIG also.) If your DNS provider uses a custom token, you can skip this part; instead, figure out how to generate the token from your DNS provider.
    1. To use TSIG authentication you need a DNSSEC key, which you can generate with the dnssec-keygen program. If you are running BIND (or a variant) as your DNS server, dnssec-keygen is probably already installed there, and it's easier just to generate your key there. If you want/need to run it on Windows you can get it from https://www.isc.org/downloads; you have to download the whole BIND zip file, but you can just extract what you want without installing anything. Extract dnssec-keygen.exe and all the .DLLs that start with "lib". (Depending on your Windows version you might also need to track down and install a Visual Studio runtime or two from Microsoft; if you do, you'll get a dialog box telling you what you need.) As a bonus you can extract the Windows ports of dig and host if you want (two common Linux DNS utilities that are quite a bit more functional than NSLOOKUP).
    2. Generate your key with the command:

      Code: Select all

      dnssec-keygen -a HMAC-SHA512 -b 512 -n HOST letsencrypt
      You can use whatever you want after the HOST directive; I just used letsencrypt so I'd remember what it was for. But whatever you use, IT HAS TO MATCH THE TSIG KEYS USED LATER ON. (This is important - don't ask me how I know.) It will generate two text files named something like Kletsencrypt.+165+12345.key and Kletsencrypt.+165+12345.private . The .private file will look something like this (not my real key, obviously):

      Code: Select all

      Private-key-format: v1.3
      Algorithm: 165 (HMAC_SHA512)
      Key: boGNa0I9EZs/KeZL+Pm2aZKmSqQsOXvi+D7JIv4RE9JkJ2ExdBGWr8gFY+XzYXtcjZ1cxTz7hagKH6vzXGmPGQ==
      Bits: AAA=
      Created: 20190417035243
      Publish: 20190417035243
      Activate: 20190417035243
      
      (Depending on your exact version of dnssec-keygen, the key may have spaces in it; these are ignored, and you can take them out in the settings below if you wish.)
    3. You need to edit your master DNS server's configuration file to allow updates using this key. On my DNS server (BIND on Ubuntu) it's in /etc/bind/named.conf.local but other systems may be slightly different (and for non-BIND-based DNS servers, you're on your own to enable remote updating). At the top of the file add the lines:

      Code: Select all

      key letsencrypt {
              algorithm hmac-sha512;
              secret "boGNa0I9EZs/KeZL+Pm2aZKmSqQsOXvi+D7JIv4RE9JkJ2ExdBGWr8gFY+XzYXtcjZ1cxTz7hagKH6vzXGmPGQ==";
      };
      
      Replace the long string in quotes after secret with the generated Key from the .private file. Three things of note here:
      • The word after key has to match what you used in the dnssec-keygen command exactly.
      • It's definitely hmac-sha512 with a dash, not an underscore (which is what the .private file shows).
      • BIND is VERY picky about semicolons.
      Then in your zone configuration section, add update-policy { grant letsencrypt zonesub TXT; }; like so:

      Code: Select all

      zone "example.com" {
              ...
      	...
              update-policy { grant letsencrypt zonesub TXT; };
      };
      Again, be careful to include all the semicolons. Restart the DNS service.

      (Side note: BIND's update-policy directive is mutually exclusive with the older allow-update directive; you can't have both in the same zone definition, even if they refer to different keys. If your zone happens to use allow-update, you'll have to choose one way or the other. update-policy allows finer-grained control.)
  4. Next create a batch file on your hMailServer machine to set the appropriate environment variables and call the Lego utility. Mine looked like this:

    Code: Select all

    @echo off
    set RFC2136_NAMESERVER=11.22.33.44
    set RFC2136_TSIG_ALGORITHM=hmac-sha512.
    set RFC2136_TSIG_KEY=letsencrypt
    set RFC2136_TSIG_SECRET=boGNa0I9EZs/KeZL+Pm2aZKmSqQsOXvi+D7JIv4RE9JkJ2ExdBGWr8gFY+XzYXtcjZ1cxTz7hagKH6vzXGmPGQ==
    
    set RFC2136_POLLING_INTERVAL=60
    set RFC2136_PROPAGATION_TIMEOUT=300
    
    REM Original cert generation:
    lego -a --email="admin@example.com" -d "example.com" -d "*.example.com" --dns rfc2136 run
    
    Items of note:
    • If your DNS provider uses a proprietary token system rather than RFC2136, you'll have different variables to set; these can be found at https://go-acme.github.io/lego/dns/.
    • FC2136_NAMESERVER is your master DNS server; apparently this must be the IP address, not the host name.
    • Note the period after hmac-sha512.
    • The value of RFC2136_TSIG_KEY has to match the name you used in the dnssec-keygen command exactly.
    • The last two environment variables are optional. Let's Encrypt appears to use Google's DNS servers to check for the TXT records, and they sometimes stop answering if you ask the same question too many times in a row. I had inconsistent results without increasing these from the defaults.
    • If you're used to using Let's Encrypt's certbot, note that Lego doesn't let you string domains together in one -d directive like certbot does. (Oops.)
  5. Run the batch file. If you have a lot of domains it will take a while; Lego rather dimly serializes the whole process, setting one TXT record and waiting for propagation before moving on to the next, rather than setting all the TXT records at once. Lego spits out a lot of diagnostics (some of which looks like problems but aren't) but at the end you should see "Validations succeeded; requesting certificates" and then "Server responded with a certificate.". Your cert and key will end up in .lego\certificates\example.com\.
  6. Next reconfigure hMailServer to point to the new cert and key. The certificate file you want is example.com.crt, NOT example.com.issuer.crt. The key file ends in .key. Bounce hMailServer, et voila, you're using your new cert.
  7. Now edit your batch file again. Mine now looks like this:

    Code: Select all

    @echo off
    set RFC2136_NAMESERVER=11.22.33.44
    set RFC2136_TSIG_ALGORITHM=hmac-sha512.
    set RFC2136_TSIG_KEY=letsencrypt
    set RFC2136_TSIG_SECRET=boGNa0I9EZs/KeZL+Pm2aZKmSqQsOXvi+D7JIv4RE9JkJ2ExdBGWr8gFY+XzYXtcjZ1cxTz7hagKH6vzXGmPGQ==
    
    set RFC2136_POLLING_INTERVAL=120
    set RFC2136_PROPAGATION_TIMEOUT=600
    
    REM Original cert generation:
    REM lego -a --email="admin@example.com" -d "example.com" -d "*.example.com" --dns rfc2136 run
    
    REM Cert renewal:
    lego -a --email="admin@example.com" -d "example.com" -d "*.example.com" --dns rfc2136 renew >>  c:\hmailserver\letsencrypt\certgen.log
    
    I commented out the original generation command and added the renewal command. I also significantly raised the polling interval and timeout, because this is now going to run unattended as a scheduled task; I want to be sure it finishes and I don't care how long it takes. I also redirect the output to a file so I can check on it if I need to.
  8. Lastly go to Task Scheduler and create a task to run the batch file once a day. Let's Encrypt certs are good for 90 days but they default to renewing in 60 days. Lego just immediately exits if it hasn't been 60 days, so there's no harm in running it that often. Don't forget to set it to run whether you're logged in or not. (Oops again.)
That's about the size of it. I apologize for the excruciating detail, but I hope it can save someone a bunch of time by not repeating my errors. Let me know if there are any mistakes in this or anything's unclear.

ras

palinka
Senior user
Senior user
Posts: 2010
Joined: 2017-09-12 17:57

Re: Lego: Windows-based auto-renewal of Let's Encrypt certs, including wildcards

Post by palinka » 2019-04-17 10:46

This is really good. Thank you!

User avatar
jimimaseye
Moderator
Moderator
Posts: 8680
Joined: 2011-09-08 17:48

Re: Lego: Windows-based auto-renewal of Let's Encrypt certs, including wildcards

Post by jimimaseye » 2019-04-17 15:42

Is it compatible for those of us running Windows DNS server (not 'bind')?

[Entered by mobile. Excuse my spelling.]
5.7 on test.
SpamassassinForWindows 3.4.0 spamd service
AV: Clamwin + Clamd service + sanesecurity defs : https://www.hmailserver.com/forum/viewtopic.php?f=21&t=26829

User avatar
ras07
Normal user
Normal user
Posts: 228
Joined: 2010-03-11 08:51

Re: Lego: Windows-based auto-renewal of Let's Encrypt certs, including wildcards

Post by ras07 » 2019-04-17 17:18

jimimaseye wrote:
2019-04-17 15:42
Is it compatible for those of us running Windows DNS server (not 'bind')?
I'm not sure. It looks like Windows DNS server supports RFC2136 but not TSIG authentication. If your hMailServer machine can authenticate through Active Directory there might be a way to do it. Or, Lego has an external program interface (see https://go-acme.github.io/lego/dns/exec/ so if there's a way to write a batch file to update your Windows DNS server from your hMailServer machine, it should be pretty easy to integrate it into Lego.

User avatar
SorenR
Senior user
Senior user
Posts: 3691
Joined: 2006-08-21 15:38
Location: Denmark

Re: Lego: Windows-based auto-renewal of Let's Encrypt certs, including wildcards

Post by SorenR » 2019-04-17 19:07

jimimaseye wrote:
2019-04-17 15:42
Is it compatible for those of us running Windows DNS server (not 'bind')?

[Entered by mobile. Excuse my spelling.]
Can you do a split horizon with MS DNS?

As I read it Lego needs to play with the public DNS to add some stuff to it.

Lego... I have fond memories of my kids playing with Lego all over the floor, not so fond memories when stepping on it in the middle of the night ... :roll:
SørenR.

“Those who don't know history are doomed to repeat it.”
― Edmund Burke

User avatar
ras07
Normal user
Normal user
Posts: 228
Joined: 2010-03-11 08:51

Re: Lego: Windows-based auto-renewal of Let's Encrypt certs, including wildcards

Post by ras07 » 2019-04-17 19:54

SorenR wrote:
2019-04-17 19:07
jimimaseye wrote:
2019-04-17 15:42
Is it compatible for those of us running Windows DNS server (not 'bind')?

[Entered by mobile. Excuse my spelling.]
Can you do a split horizon with MS DNS?

As I read it Lego needs to play with the public DNS to add some stuff to it.

Lego... I have fond memories of my kids playing with Lego all over the floor, not so fond memories when stepping on it in the middle of the night ... :roll:
I believe split horizon is supported as of at least Windows Server 2016; not sure about before that.

Lego (actually any Let's Encrypt client that supports DNS authentication) does have to manipulate the public DNS records.

Legos (the bricks) have barefoot-seeking technology.

ras

User avatar
ras07
Normal user
Normal user
Posts: 228
Joined: 2010-03-11 08:51

Re: Lego: Windows-based auto-renewal of Let's Encrypt certs, including wildcards

Post by ras07 » 2019-08-20 23:06

IMPORTANT UPDATE:

TL;DR: add -k rsa4096 to each of the lego commands in the above scripts to support encryption of email coming in from Outlook and Hotmail users.

DETAILS: After running this for a few months, I noticed that I was getting more "no shared cipher" errors than I used to. Most of the IPs in question originated from Russia and Brazil, which I assume were spam. However a lot of them were Outlook IPs. After some research I believe I was seeing the same problem as this thread from 2.5 years ago (the difference being that back then email from Outlook apparently just failed outright; nowadays, although Outlook still doesn't understand EC certs, it at least retries the session without STARTTLS, so it eventually does send the email - just unencrypted). I also found a couple other (legit) senders over the months that never did retry without STARTTLS (so those emails just failed to go through at all).

To summarize the above thread, apparently Outlook's MTAs (and a few other senders) don't understand Elliptic Curve certs. The Lego client (among others) defaults to EC. There's an easy fix: just add the -k rsaXXXX flag to the lego command line (where XXXX is the key length; I used rsa4096, but rsa2048 and rsa8192 are other options - although I didn't validate which key lengths Outlook understands other than 4096).

So in the above instructions, these two lines:

Code: Select all

lego -a --email="admin@example.com" -d "example.com" -d "*.example.com" --dns rfc2136 run
...
lego -a --email="admin@example.com" -d "example.com" -d "*.example.com" --dns rfc2136 renew >>  c:\hmailserver\letsencrypt\certgen.log
should become

Code: Select all

lego -a -k rsa4096 --email="admin@example.com" -d "example.com" -d "*.example.com" --dns rfc2136 run
...
lego -a -k rsa4096 --email="admin@example.com" -d "example.com" -d "*.example.com" --dns rfc2136 renew >>  c:\hmailserver\letsencrypt\certgen.log
If you've already generated (default) EC certs, you should be able to just renew the cert with the -k rsa4096 flag; you can force immediate renewal (regardless of when the cert expires) by adding --days 99 to the lego command line (this switch must appear after the renew option). So replace this line:

Code: Select all

lego -a --email="admin@example.com" -d "example.com" -d "*.example.com" --dns rfc2136 renew >>  c:\hmailserver\letsencrypt\certgen.log
with this:

Code: Select all

lego -a -d rsa4096 --email="admin@example.com" -d "example.com" -d "*.example.com" --dns rfc2136 renew --days 99 >>  c:\hmailserver\letsencrypt\certgen.log
(if you edit the renewal batch file that you have scheduled in Task Scheduler, don't forget to edit it back to get rid of the --days 99 flag, or else it will renew every day.) Alternately, to start completely over you can erase/rename the .lego directory and rerun the original generating scripts.

It appears that multiple LetsEncrypt clients now default to ECC certs - so regardless of what client you use, if you're using LetsEncrypt to generate certs for hMailServer (or any other mail server), you probably need to figure out how to make your client generate RSA certs rather than ECC, in order to insure compatibility with Outlook/Hotmail (and a few other oddball MTAs).

ras

Post Reply