How to get SSL/TLS certificates at UCC for that sweet HTTPS (and other) goodness.
Since the launch of the free SSL/TLS certificate authority Let's Encrypt, UCC has been saving money and SSLising everything by using it.
LE was (is?) revolutionary in that it is free and completely automated. The exact method you want to use depends on which machine and which service you are getting a certificate for. We use the official Certbot client (UCC systems were previously set up with acmetool, but this is no longer maintained and the official client is fine now). The certificates are automatically renewed every two months as they are only good for three months.
On new machines, install certbot by using apt install certbot.
If the host is reachable by HTTP/HTTPS after TheCloudflarening, Letsencrypt HTTP challenges can be used.
You will probably also want one of the following:
python3-certbot-apache (for automatic verification and setup with Apache)
python3-certbot-nginx (for automatic verification and setup with Nginx)
If you don't have an existing webserver setup that you are worried about breaking, you can use the Fully Automatic method (apache shown below, can also use nginx):
# apt install certbot python3-certbot-apache # Create an account - use your personal email here if this is not an official UCC server certbot register --agree-tos -m [email protected] # Using the apache installer, get certificates for somehost in both usual domains certbot run --apache --domains somehost.ucc.asn.au,somehost.ucc.gu.uwa.edu.au
That's it! You're done. The Debian packages set up automatic renewal and reinstallation of the certificates.
Existing web servers (eg Mussel)
Although we have good backups and version control, nobody has ever tested whether the core webservers handle the automagic configuration method. If it breaks, you get to put the pieces back together.
If you aren't willing to be the first, or you need the certificate for another service (eg mail server) you can use the certonly command with the preconfigured webroot, like this:
certbot certonly --webroot --webroot-path /services/ssl/acme-challenge/ -d cooluserhostingatucc.id.au,www.cooluserhostingatucc.id.au
/services/ssl/acme-challenge is set up on Mussel and Mooneye so that the various redirects and firewalls don't make it too hard to get the right certificates. Otherwise, use a webroot that you have configured in the server software such as:
# cat /etc/apache2/conf-available/letsencrypt.conf Alias "/.well-known/acme-challenge/" "/var/www/.well-known/acme-challenge/" <Directory "/var/www/.well-known/acme-challenge/"> AllowOverride None Options None Require all granted </Directory> # a2enconf letsencrypt
certbot certificates will show you the paths you need for the Apache config.
While the automatic issuers know how to restart the services you are interested in, certbot certonly does not. You may need to create a restart hook, which lives in /etc/letsencrypt/renewal-hooks/deploy/. It can be as simple as systemctl reload apache2, or much more complicated. Ask for help if you need it.
DNS challenges and wildcard certificates
Danger Will Robinson
Improperly configured DNS or disclosure of dynamic DNS update keys can lead to domain compromise. Don't hesitate to ask for help if you are not sure.
Most of the time, Let's Encrypt/certbot will use a Web server to prove that the machine owns the domain it is using. Sometimes this is impossible (due to firewalling, wildcard certificates, multiple machines sharing a hostname). If necessary, DNS verification can be used.
Previously UCC used RFC2136 DNS updates to assist with DNS verification. This no longer works post TheCloudflarening. We have to either use a Cloudflare API key, or delegate the DNS challenges and use a helper to update the DNS appropriately.
Wildcard certs using Cloudflare API key
These are currently being used on mailfish, mussel, and motsugo. Primarily, hosts with large numbers of virtual hosts. As this uses the Cloudflare API, this should be used on trusted hosts only where absolutely necessarily. It is better to use the alternatively below in most other circumstances.
These hosts have been configured using the https://gitlab.ucc.asn.au/ucc-systems/ucc-ansible-soe UCC Ansible SOE. This contains a ucc_wildcard_cert role.
The role does several things
- If the version of certbot is older than 1.6, build a more recent version
- Installs python3-certbot-dns-cloudflare plugin
- Grabs the Cloudflare API key out of uccpass
- Places the credentials on the host
- Runs certbot to generate the cert
certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/dns-cloudflare-credentials.ini --email [email protected] --server https://acme-v02.api.letsencrypt.org/directory --agree-tos -d ucc.asn.au -d ucc.gu.uwa.edu.au -d *.ucc.asn.au -d *.ucc.gu.uwa.edu.au
Non-Wildcard certs using Quovadis API helper
As an alternative to using the Cloudflare API key, a REST-like API helper has been created to update the delegated DNS text records to perform the ACME challenge.
The quovadis tool can be accessed at https://quovadis.ucc.asn.au/quovadis/
- Provides an API to update _acme_challenge DNS entries (delegated via a CNAME in the ucc.machines / zonemake.py setup)
- Uses desec.io to host a zone that can be updated via an API for the challenges
- Has an example certbot helper script
The code for the Quovadis tool itself can be found in the UCC gitlab, https://gitlab.ucc.asn.au/UCC/quovadis/. You shouldn't have to touch it in normal circumstances.
Quovadis has a relatively minimal API. The username used by the API is your UCC username.
curl -X POST -d username=accmurphy https://quovadis.ucc.asn.au/quovadis/register
This will cause your API key to be sent to your UCC email address. For UCC core systems, use "root". The API key for "root" is stored in uccpass.
It is possible to update the API key via an API call.
curl -X POST -d username=accmurphy -d api_key=foo -d new_api_key=bar https://quovadis.ucc.asn.au/quovadis/update-api-key
curl -X POST -d username=accmurphy -d api_key=foo -d challenge=thingy -d value=123456abcdef https://quovadis.ucc.asn.au/quovadis/update-challenge
The easiest way to use the API is by a certbot hook script.
certbot certonly --server https://acme-v02.api.letsencrypt.org/directory \ --manual \ --preferred-challenges=dns \ --manual-auth-hook /etc/letsencrypt/ucc-hooks/certbot-hook-quovadis.sh \ -d thehostname.ucc.asn.au
The hook script, and an example config file, can be downloaded from:
mkdir /etc/letsencrypt/ucc-hooks cp certbot-hook-quovadis.sh /etc/letsencrypt/ucc-hooks
To configure, create a config file config.sh in the same directory as the script containing
QV_API_KEY=<your API key> UCC_USERNAME=<your UCC username>
A DNS delegation must be setup for the ACME challenge. In ucc.machines, this looks like:
[ipsec] A: 1 zones: ucc asn [_acme-challenge.ipsec] CNAME: root-ipsec.quovadis-challenges.ucc.asn.au. zones: ucc asn
In this example ipsec is the relevant hostname (corresponding to the challenge argument in the API), root is the username
To use the script, runs it as a manual auth hook with certbot. An example command line is:
certbot certonly --server https://acme-v02.api.letsencrypt.org/directory \ --manual \ --preferred-challenges=dns \ --manual-auth-hook /etc/letsencrypt/ucc-hooks/certbot-hook-quovadis.sh \ -d example.ucc.asn.au,example.ucc.gu.uwa.edu.au
- Automatic certbot renew renewals may fail until they have been manually run once with: certbot renew --manual-public-ip-logging-ok
- This setting is recorded in /etc/letsencrypt/renewal/mudkip.ucc.asn.au.conf
- manual_public_ip_logging_ok = True
Post certificate generation things
Run certbot certificates to see it again.
Don't forget to set up a reload hook. Here is one for strongSwan in /etc/letsencrypt/renewal-hooks/deploy/reload-strongswan:
[ -f /run/starter.charon.pid ] && kill -s USR1 $(cat /run/starter.charon.pid) && /usr/sbin/ipsec rereadall
For a wildcard certificate, just add or change the --domains argument above (eg --domains secure.ucc.asn.au,\*.secure.ucc.asn.au). Be very careful with the key material that is generated, as wildcard certificates pose extra security risks.