Private Docker Registry 'x509: certificate signed by unknown authority'

While setting up a new private docker image registry with certificates signed by an internal certificate authority this week we ran into an issue getting our docker nodes to communicate:

Error response from daemon: Get https://private.registry.tld/v2/: x509: certificate signed by unknown authority

Following the guidance on self-signed certificates from Docker did not directly address the issue.

References

• Use Self Signed Certificates [docs.docker.com]
• Support for intermediate certificates [github.com/docker]
• Include configuration explanation for intermediate TLS certificates [github.com/docker]
• Error response from daemon: Missing certificate domain.cert for key domain.key [github.com/docker]
• Documentation on setting up certificates in docker engine should not have been removed [github.com/moby]

Error Messages

• Error response from daemon: missing key ca.key for client certificate ca.cert. Note that CA certificates should use the extension .crt
• Error response from daemon: Get https://private.registry.tld/v2/: x509: certificate signed by unknown authority

While investigating these errors we discovered a few things about pinning certificates to custom private image registries in Docker:

• How you name your ca certificate matters:
  • ca.crt should be the CA certificate (and intermediate root certificates concatenated as well, if any)
  • client.cert and client.key should be used for client certificate based authentication
  • If you name your CA certificate something else it may not work

Solution

In our case we found that while we used the correct root certificate, we were not given the correct intermediate root certificates. Once we had those and concatenated them together (with the Root CA as the first cert, intermediates following as chained) and named the resulting file ca.crt the problem went away.

For reference, to get a custom root certificate to be recognized by docker you must create a folder with the name of your registry (whether it be by IP address or DNS Name) and place the certificate beneath it like so:

/etc/docker/certs.d/name.or.ip.of.registry/ca.crt ** This needs to be done on every docker host that needs to connect to the registry

With more recent versions of docker you DO NOT have to restart the docker daemon for the changes to take effect (tested on 18.09 docker release)