[FIXED] How can I securely push from a GitLab Runner KubernetesExecutor pod to a private container registry?

Issue

Goal

Build a CI/CD pipeline multiple GitLab repositories with a certain project structure can make use of. For this, a Docker container with Python code is built and subsequently securely pushed to Google Cloud's Container Registry.

Set up

• KubernetesExecutor is installed on Kubernetes Engine using the Helm chart as provided by GitLab.
• The base image for the build process (runners.image in the values.yaml) is a custom one as this helps automatically containerising the provided repository. The reason this is worth mentioning is that this is from the same private repository as where the image should be pushed to.
• Right now, building the container from the repository runs successfully (see code below).

Problem

How can I push the image to the Container Registry without adding a service account key to a Docker image (otherwise, please convince me this isn't bad practice)?

Code

.gitlab-ci.yml

services:
  - docker:19.03.1-dind

stages:
  - build

build:
  stage: build
  script:
    - docker build -t ${CONTAINER_REGISTRY}/pyton-container-test:latest .
    # This line is where I'd need to use `docker login`, I guess.
    - docker push ${CONTAINER_REGISTRY}/python-container-test:latest

values.yaml (Helm)

It's worth mentioning that the following environment variables are set by the GitLab Runner:

runners:
  env:
    DOCKER_DRIVER: overlay2
    DOCKER_HOST: tcp://localhost:2375
    DOCKER_TLS_CERTDIR: ""
    CONTAINER_REGISTRY: eu.gcr.io/<project_id>

Direction of solution

I think I should be able to mount a secret from the Kubernetes cluster to the GitLab Runner build pod, but I can't seem to find a way to do that. Then, I should be able to add the following line into .gitlab-ci.yml:

cat mounted_secret.json | docker login -u _json_key --password-stdin https://eu.gcr.io

Setting up config.toml to use a secret volume should work. However, with a Helm chart this doesn't seem possible yet.

Notes

• It is possible to set protected environment variables in GitLab CI, but I'd rather not, as they're harder to maintain.
• I've investigated this answer, but this says I need to add a key to my Docker image.
• Looked into the GitLab documentation on using a private container registry, but don't seem to get much further with that.
• A similar problem would occur when, for example, it must connect to a database during the build process.

Solution

It's not possible with the default Helm chart as provided by GitLab. However, there is a workaround when you customise theirs.

In templates/configmap.yaml, it's possible to edit the entrypoint. At the very end, the runner is started as follows:

# Start the runner
exec /entrypoint run --user=gitlab-runner \
     --working-directory=/home/gitlab-runner

For this to work, a config.toml is generated based upon the provided values.yaml (with the right Runner token). This means that right before this step, we could edit the config.toml to our needs. In my case, I simply added:

echo "    [[runners.kubernetes.volumes.secret]]" >> ${CONFIG_FILE}
echo "      name = \"{{ .Values.secretName }}\"" >> ${CONFIG_FILE}
echo "      mount_path = \"/keys\"" >> ${CONFIG_FILE}
echo "      read_only = true" >> ${CONFIG_FILE}

Where ${CONFIG_FILE} is /home/gitlab-runner/.gitlab-runner/config.toml.

Finally, you can deploy your GitLab Runner using:

$ helm install project_name -f values.yaml <path to chart>

Answered By - bartcode

Answer Checked By - Mildred Charles (PHPFixing Admin)