Semi-automatic Let's Encrypt Renewal for Gitlab Pages Redux

Reading time ~3 minutes

Deprecated

Look Here

Back in late 2016 I wrote a fragile expect script. It was an attempt to automate certificate renewal for the free Let’s Encrypt service when used with GitLab pages. Let’s just say it sucked and I’m sorry to anyone who ever attempted to use it, let alone those who tried to reach me for support. I couldn’t reproduce their issues. I could fix their bugs. And I generally sucked at responding.

Then the certbot rev’ed (I’m at 0.21.0) and some of the text output changed. It broke the expect script well and truly. That makes for a perfect time to explore a different solution.

After some digging I found the --manual-auth-hook command line option for certbot. I don’t know how I missed it before, or whether it wasn’t there in 0.19.0. Clearly, if I could make a script to be driven by certbot then I could get away from trying to wrap it in something else.

This resulted in the Radically Simplified Bash Script® to automate publishing certbot authorization files.

Configuration

The setup for the GitLab CI/CD is the same as before.

There are two pieces of configuration necessary. The first is the creation of a Certbot .ini file. It specifies your domains, your email address, and directories that Certbot will use. Oddly, it seems you can not specify where the certificates will be stored. That remains under /etc/letsencrypt/.

Example .ini file below:

text = True
domains = [comma delimited list of domains]
email = [your email address]
renew-by-default
agree-tos
rsa-key-size = 4096
logs-dir = /Users/andy/tmp/certbot/logs/
work-dir = /Users/andy/tmp/certbot/work/

Note that the logs-dir and work-dir need to be created and owned by your user account and not root. The ssl-renew.ex script does not run as root, but should be run as the local user..

You will need to modify the gitlab-auth-hook.sh script to let it know where various files and directories reside. The values in question are:

  • SaveDir – Where to save the Let’s Encrypt validation files within your repository.
  • CertPath – The URI directory where the nounce files will be published and available via HTTP.

Example:

SaveDir="/home/andy/code/gitlab//autonomic-guru/le"
CertPath="/.well-known/acme-challenge/"

Running

$ certbot certonly --config autonomic-guru.ini --manual-auth-hook gitlab-auth-hook.sh --manual --preferred-challenges=http -n --manual-public-ip-logging-ok

This could take a while depending on the number of SANs (domains) in your request and how performant the GitLab CI/CD pipeline is that day. I’ve had this take hours because of GitLab backend issues.

The following is an example output from my last renewal.

$ certbot certonly --config autonomic-guru.ini --manual-auth-hook ./gitlab-auth-hook.sh --manual --preferred-challenges=http -n --manual-public-ip-logging-ok
Saving debug log to /Users/u309/tmp/certbot/logs/letsencrypt.log
Plugins selected: Authenticator manual, Installer None
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for autonomic.guru
http-01 challenge for www.autonomic.guru
Waiting for verification...
Cleaning up challenges

IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at:
   /etc/letsencrypt/live/autonomic.guru/fullchain.pem
   Your key file has been saved at:
   /etc/letsencrypt/live/autonomic.guru/privkey.pem
   Your cert will expire on 2018-04-21. To obtain a new or tweaked
   version of this certificate in the future, simply run certbot
   again. To non-interactively renew *all* of your certificates, run
   "certbot renew"
 - If you like Certbot, please consider supporting our work by:

   Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
   Donating to EFF:                    https://eff.org/donate-le

As you can see, far more quiet than before. But if the renewal is just sitting there, you might wish to see what’s happening on the GitLap side. In the GitLab project UI, you will be able to see each Certbot validation commit and subsequent CI build.

/images/posts/2018-01-21-gl-pipeline.png

Updating Your Certs

Sadly, nothing has changed here on the GitLab side. Bring on the manual part.

At this time there is no API call available to add/remove/update a domain in GitLab Pages. Additionally, you can’t update an existing domain to change the certificate.

The process to update is similar to the steps to add the certificates when you first enabled SSL to your GitLab Pages. Get your certificate and private key ready (found in /etc/letsencrypt/live/[domain]/). Then steps are:

  1. Go to the Pages section of the GitLab Project Web UI.
  2. /images/posts/2016-11-remove.png your domain (I really hate this).
  3. Add your domain /images/posts/2016-10-28-gitlab-new-domain.png (well, new again).
  4. Use the fullchain.pem for the Certificate text box and privkey.pem in the Key text box.
  5. Then click /images/posts/2016-11-create-new-domain.png.
  6. Should only take about a minute or two for the domain and certificate to update and you can confirm it in your browser.

/images/posts/2016-11-cert-validation.png

Additional Notes

  • Make sure you’re working on the branch that GitLab pages will publish on push (typically master)
  • I regularly remove certbot files from the le directory in my repo.
     git rm 6jAZH_fKlUQLd_Ar20HtNg4is3DIEjSPIx9p6iF1gGw
     git commit -a -m "removed file"
     git push
    
  • GitLab’s CI/CD pipeline doesn’t like empty directories, so I keep at least one file in the le directory.
     touch place_holder
     git add place_holder
     git commit -m "a place holder" place_holder
     git push
    

Leaving Apple: part 1

## Leaving the MacBook...The choice by Apple to abandon "legacy" interfaces (USB-A, analogaudio, Ethernet, HDMI, MagSafe, etc) in favor o...… Continue reading

Bad Day on the Road

Published on June 24, 2019

piTravel Clock Build

Published on November 24, 2018