We have some users who are trying to push Docker containers in to a Gitlab registry and their push is being rejected because of an invalid certificate.  This was working last week before doing yum update, upgrading from Gitlab 10.2.x, and enabling HTTPS on the Gitlab web interface using WeEncrypt certificates.

We’re running the following software versions on the Gitlab server

Gitlab CE 10.5.2
Comodo Wildcard Certificate
CentOS 6.9

Client workstation

CentOS 7.4
Docker CE 17.20.0

and attempting to do docker login with

 docker login -u jeff@example.com -p PASSWORD registry.example.com:5050

This is the error the developer was getting:

Error response from daemon: Get https://registry.example.com:5050/v2/: Get https://gitlab.example.com/jwt/auth?account=jeff%40example.com&client_id=docker&offline_token=true&service=container_registry: x509: certificate signed by unknown authority

There was a secondary issue as well that started happening, normal users trying to check out code were now forced to use SSL which is what we want

git clone https://gitlab.example.com/project/mysite-web.git

however, they are where getting an error and not able to successfully clone the repo

fatal: unable to access 'https://gitlab.example.com/project/mysite-web.git/': Peer's Certificate issuer is not recognized.

So I assumed it was the WeEncrypt certificate not working for some reason, and replaced it with our wildcard certificate. I did some tests using curl

curl -l -v  https://registry.example.com:5050

which looks something like this:

* About to connect() to gitlab.example.com port 443 (#0)
*   Trying
* Connected to gitlab.example.com ( port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
* Server certificate:
* 	subject: CN=*.example.com,OU=PositiveSSL Wildcard,OU=Domain Control Validated
* 	start date: Sep 19 00:00:00 2017 GMT
* 	expire date: Sep 19 23:59:59 2018 GMT
* 	common name: *.example.com
* 	issuer: CN=COMODO RSA Domain Validation Secure Server CA,O=COMODO CA Limited,L=Salford,ST=Greater Manchester,C=GB

It looks okay, but not completely working yet.

Basically it seems that the SSL certificate settings on Gitlab needed to be changed.  I need to make sure both the registry and the repo were using a pem file.

Placed all my files in /etc/gitlab/ssl


 Then created a pem file by combining the ca and crt

cat __example.com.crt __example.com.ca-bundle > star.example.com.pem

Now I updated the configuration for Gitlab server

vi /etc/gitlab/gitlab.rb

Ensure you have the following lines, I added mine at the bottom after all the commented out examples just so I can see all my settings in a common location

external_url 'https://gitlab.example.com'
nginx['redirect_http_to_https'] = true
nginx['redirect_http_to_https_port'] = 80
nginx['ssl_certificate'] = "/etc/gitlab/ssl/star.example.com.pem"
nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/star.example.com.key"

registry['enable'] = true
registry_external_url 'https://registry.example.com:5050'
gitlab_rails['registry_api_url'] = "http://localhost:5000"
registry['registry_http_addr'] = "localhost:5000"
registry_nginx['ssl_certificate'] = "/etc/gitlab/ssl/star.example.com.pem"
registry_nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/star.example.com.key"

Next reconfigure Gitlab settings

gitlab-ctl reconfigure

Then restart the two services we modified

gitlab-ctl restart registry
gitlab-ctl restart nginx

Now test by running the docker login and git clone command again.

# docker login -u jeff@example.com -p PASSWORD registry.example.com:5050
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Login Succeeded

# git clone https://gitlab.example.com/project/mysite-web.git
Cloning into 'mysite-web'...
Username for 'https://gitlab.example.com': jeff@example.com
Password for 'https://jeff@example.com@gitlab.example.com':
remote: Counting objects: 86729, done.
remote: Compressing objects: 100% (24177/24177), done.
Receiving objects:  83% (71986/86729), 109.71 MiB | 13.37 MiB/s

Looks like the both work, problem fixed.