Renewing the (Default) PMM Certificate

Recently one of our colleagues was struggling with the default self-signed SSL certificate that ships with Percona Monitoring and Management (PMM). He needed an older version of PMM to run in a test environment for a particular project. He was using Docker to run the PMM container, however when the container started up, the SSL certificate configured in Nginx was already expired. MacOS and Google Chrome didn’t want to import the expired certificate in the trust store, so we couldn’t access the PMM web UI. 

This challenge led me to help him recreate the certificate to unblock his project. So, let’s get started.

Disclaimer: you should not use the default certificate for anything else other than for testing on your local machine. As soon as you’re thinking about running Percona Monitoring and Management  (PMM), you should configure a valid certificate signed by a certification authority (CA) that you trust. Your company might already be running its own CA. If not, you can always fall back to an external CA, such as  a non-profit, open CA like Let’s Encrypt (www.letsencrypt.org).

Next, you need to connect to your container to access the files. This command gives  you access to a terminal inside the container:

$ docker exec -ti pmm-server /bin/bash                                                                                            
[root@7f7977c9228a opt]#

PMM stores the certificates for PMM in the /srv/nginxdirectory inside the container:

[root@7f7977c9228a opt]# ls -hl /srv/nginx/certificate.*
total 28K
-rw-r--r-- 1 root root  137 Oct 19  2020 certificate.conf
-rw-r--r-- 1 root root  977 Oct 19  2020 certificate.crt
-rw-r--r-- 1 root root 1.7K Oct 19  2020 certificate.key

The certificate.crt file is the actual certificate used by Nginx, the certificate.key file contains the private key used by the SSL certificate. Make sure to keep this key secure. If you want to create your own certificate you should also generate a new key for it.

The third file in this folder is the configuration file used to generate the original certificate shipped with the container. This helps to generate the exact same certificate currently in use:

[root@7f7977c9228a opt]# cat /srv/nginx/certificate.conf

[ req ]
distinguished_name = req_distinguished_name
prompt             = no

[ req_distinguished_name ]
O                  = Main Org.

First, a certificate request. I’ll  use openssl to create the request:

[root@7f7977c9228a opt]# openssl req \  # use the req subcommand to create request
  -new \                                # create a new request
  -key /srv/nginx/certificate.key \     # re-use the existing key
  -config /srv/nginx/certificate.conf \ # use the provided config
  -out /srv/nginx/certificate.csr       # write the request to a file

[root@7f7977c9228a opt]# ls -hl /srv/nginx/certificate.csr
-rw-r--r-- 1 root root 891 Dec 15 18:41 /srv/nginx/certificate.csr
[root@7f7977c9228a opt]#

Next, sign the certificate request. This can be done by a trusted CA or you can sign the request yourself. However, in the latter case, the certificate won’t be trusted by your browser by default. Keep this in mind as it might cause alerts or issues in your browser.

To self-sign the certificate, use the following command:

[root@7f7977c9228a opt]# openssl x509 \ # use the x509 subcommand, to handle certs
  -req \                                # input is a cert request, sign and output
  -in /srv/nginx/certificate.csr \      # input is the request file
  -signkey /srv/nginx/certificate.key \ # we self-sign the cert with our own key
  -out /srv/nginx/certificate.crt       # output certificate to a file

Signature ok
subject=/O=Main Org.
Getting Private key
[root@7f7977c9228a opt]#

By default, the certificate remains signed  for 30 days. This will make you repeat this procedure every month, so if you add the parameter -days 1000 you can sign the certificate for a period of about 3 years.

We now have a new crt file to use with Nginx:

[root@7f7977c9228a opt]# ls -hl /srv/nginx/certificate.*
-rw-r--r-- 1 root root  137 Oct 19  2020 /srv/nginx/certificate.conf
-rw-r--r-- 1 root root  977 Dec 15 18:43 /srv/nginx/certificate.crt
-rw-r--r-- 1 root root  891 Dec 15 18:41 /srv/nginx/certificate.csr
-rw-r--r-- 1 root root 1.7K Oct 19  2020 /srv/nginx/certificate.key
[root@7f7977c9228a opt]#

The only remaining step is to restart Nginx to start using the new certificate. PMM uses supervisord to manage the different applications it runs:

[root@7f7977c9228a opt]# supervisorctl restart nginx
nginx: stopped
nginx: started
[root@7f7977c9228a opt]#

Et voilà. A fresh self-signed certificate to use for all your testing purposes:

Renewing the default self-signed certificate for PMM is very simple. However, you should be warned that running a self-signed certificate will still require changes to the browser certificate trust store. This is not recommended for anything more than some testing purposes.

Feel free to drop any questions in the comments and don’t forget to sign up for the next post.