Setup a Gitlab Runner with docker-in-docker Support
Personally, I prefer to set up my own Gitlab Runners. The reasons being that I don’t have to wait for a runner to become available and they’re usually faster then the Gitlab Shared Runners.
Assumptions
This guide assumes you have a system with Ubuntu 18.04 which is permanently connected to the internet. A very cheap and easy way to do this is using Vultr. (Use my affiliate link and get $50 to test their platform.)
Optional: Enable Automatic Updates
As I am a lazy person, I usually enable automatic updates on my Gitlab Runners. Sometimes an update may break things, but most of the time they run smoothly.
# Install Unattended Upgrades package
sudo apt update
sudo apt install unattended-upgrades
# Configure Unattended Upgrades
cat << EOF | sudo tee /etc/apt/apt.conf.d/50unattended-upgrades
# Enable Upgrade for all package types
Unattended-Upgrade::Allowed-Origins {
"${distro_id}:${distro_codename}";
"${distro_id}:${distro_codename}-security";
"${distro_id}ESM:${distro_codename}";
"${distro_id}:${distro_codename}-updates";
"${distro_id}:${distro_codename}-proposed";
"${distro_id}:${distro_codename}-backports";
};
Unattended-Upgrade::DevRelease "false";
# Cleanup
Unattended-Upgrade::Remove-Unused-Kernel-Packages "true";
Unattended-Upgrade::Remove-Unused-Dependencies "true";
# Reboot during the night
Unattended-Upgrade::Automatic-Reboot "true";
Unattended-Upgrade::Automatic-Reboot-Time "02:00";
EOF
# Enable Unattended Upgrades
cat << EOF | sudo tee /etc/apt/apt.conf.d/20auto-upgrades
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Download-Upgradeable-Packages "1";
APT::Periodic::AutocleanInterval "7";
APT::Periodic::Unattended-Upgrade "1";
EOF
Install Docker
The next step is to install Docker:
# Remove any version of Docker that came with your distro
sudo apt remove docker docker-engine docker.io
# Install secure transport
sudo apt install apt-transport-https ca-certificates curl software-properties-common
# Add the GPG key for the Docker repository
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
# Add the Docker repository
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
# Install Docker
sudo apt update
sudo apt install docker-ce
# Optional: Add yourself to the Docker group
sudo usermod -aG docker $USER
Install the Gitlab Runner
Now it’s time to install the Gitlab Runner.
# Install the repository
curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh | sudo bash
# Install the gitlab-runner package
sudo apt install gitlab-runner
# Install the Docker cleanup script
cat << EOF | sudo tee /usr/bin/clear-docker-cache
#!/bin/sh
# (c) Gitlab
# https://gitlab.com/gitlab-org/gitlab-runner/blob/master/packaging/root/usr/share/gitlab-runner/clear-docker-cache
set -e
docker version >/dev/null 2>/dev/null
echo Clearing docker cache...
CONTAINERS=\$(docker ps -a -q \\
--filter=status=exited \\
--filter=status=dead \\
--filter=label=com.gitlab.gitlab-runner.type=cache)
if [ -n "\${CONTAINERS}" ]; then
docker rm -v \${CONTAINERS}
fi
EOF
sudo chmod a+x /usr/bin/clear-docker-cache
Register the Gitlab Runner
The next step is to register the Gitlab runner with your Gitlab group/project. You get the token to register the runner when you go to your group/project, then Settings and then CI / CD. On that page expand Runners. Then there’s a section called Set up a group Runner manually where the token is displayed.
$ sudo gitlab-runner register
Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com )
https://gitlab.com
Please enter the gitlab-ci token for this runner
ABCabc_ABCabc
Please enter the gitlab-ci description for this runner
[hostname] myserver
Please enter the gitlab-ci tags for this runner (comma separated):
docker,linux
Please enter the executor: ssh, docker+machine, docker-ssh+machine, kubernetes, docker, parallels, virtualbox, docker-ssh, shell:
docker
Please enter the Docker image (eg. ruby:2.1):
alpine:latest
Enable Docker-in-Docker
The enable docker-in-docker some changes to the gitlab runner configuration are required.
cat << EOF | sudo patch /etc/gitlab-runner/config.toml
--- /etc/gitlab-runner/config.toml
+++ /etc/gitlab-runner/config.toml
@@ -9,15 +9,16 @@
url = "https://gitlab.com/"
token = "xyzXYZxyzXYZ"
executor = "docker"
+ environment = ["DOCKER_TLS_CERTDIR=/certs","DOCKER_DRIVER=overlay2"]
[runners.custom_build_dir]
[runners.docker]
tls_verify = false
image = "alpine:latest"
- privileged = false
+ privileged = true
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
- volumes = ["/cache"]
+ volumes = ["/certs/client", "/cache"]
shm_size = 0
[runners.cache]
[runners.cache.s3]
EOF
Enable the Gitlab Runner
Since we changed the configuration, we need to restart the Gitlab runner. This is a good chance to ensure that it loads automatically when the system starts.
sudo systemctl enable docker gitlab-runner
sudo systemctl restart docker gitlab-runner
Use the Gitlab Runner in your jobs
When you want to use your runner, you can either add docker to the tags section of your .gitlab-ci.yml.
Or you can configure the Gitlab CI runner to be used for any job.
If you want that, go to the CI / CD section in the Settings of your group/project. Again, expand the Runners section. Here you should find your runner. Now on the pencil icon next to it.
On this page you have to tick the checkbox Indicates whether this runner can pick jobs without tags.
Don’t forget to Save changes.
Now, all your ci jobs will be executed by your private Gitlab runner 🚀.