Low-RAM GitLab in Docker behind NGINX proxy

update

Update 2024-05-20

GitLab 17.0.0 comes with a breaking change within the sidekiq component: the concurrency property has been changed to concurrency. This post has been updated accordingly. For more information, see their deprecation warning and the updated documentation.

GitLab, a comprehensive code management platform, extends Git functionalities with features like issue management, pull requests, teams, workflows, and more, akin to GitHub. Unlike GitHub, GitLab also offers its open source version that can be self-hosted. In this post, we will explore how to run a customized GitLab instance inside a Docker container.

By default, GitLab is configured for high-performance environments. However, running it on a VPS in our home might lead to excessive RAM usage. To address this, GitLab provides a guide on setting up an instance in a memory-constrained environment.

Let’s delve into implementing this in our Dockerized GitLab. We’ll cover how to set up the instance when using it behind an NGINX reverse proxy (in its container) and also how to configure a self-hosted GitHub runner for our GitLab instance, all within Docker containers.

GitLab base setup

The Compose file I use, with adjustments for reducing memory usage, is as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

version: '3.6'
services:
  web:
    image: 'gitlab/gitlab-ee:latest'
    container_name: gitlab
    restart: always
    environment:
      GITLAB_OMNIBUS_CONFIG: |
        external_url 'https://gitlab.edotm.net'
        gitlab_rails['gitlab_shell_ssh_port'] = 3022
        nginx['listen_https'] = false
        nginx['listen_port'] = 80
        letsencrypt['enable'] = false
        puma['worker_processes'] = 0
        sidekiq['concurrency'] = 5
        prometheus_monitoring['enable'] = false        
      # Other gitlab.rb configuration
    ports:
      - '3022:22'
    volumes:
      - './gitlab/config:/etc/gitlab'
      - './gitlab/logs:/var/log/gitlab'
      - './gitlab/data:/var/opt/gitlab'
    shm_size: '256m'

Let’s walk through it:

In the guide, the GitLab Team also proposes other configuration options, but they didn’t work in my case, and I was happy with the memory reduction up to 3x.

feedback

Important

Keep in mind that this configuration is intended for use with a reverse proxy.

GitLab behind NGINX

To make GitLab work behind a containerized NGINX reverse proxy, the configuration changes a bit (the changes are highlighted):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

version: '3.6'
services:
  web:
    image: 'gitlab/gitlab-ee:latest'
    container_name: gitlab
    restart: always
    hostname: gitlab
    environment:
      GITLAB_OMNIBUS_CONFIG: |
        external_url 'https://gitlab.edotm.net'
        gitlab_rails['gitlab_shell_ssh_port'] = 3022
        nginx['listen_https'] = false
        nginx['listen_port'] = 80
        letsencrypt['enable'] = false
        puma['worker_processes'] = 0
        sidekiq['concurrency'] = 5
        prometheus_monitoring['enable'] = false        
      # Other gitlab.rb configuration
    ports:
      - '3022:22'
    volumes:
      - './gitlab/config:/etc/gitlab'
      - './gitlab/logs:/var/log/gitlab'
      - './gitlab/data:/var/opt/gitlab'
    shm_size: '256m'
    networks:
      - nginx-net

networks:
  nginx-net:
    external: true

The hostname is the name with which GitLab will be reachable to other containers, namely NGINX. The nginx-net is the network that we created in the last blog post.

The NGINX configuration is the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

server {
  listen 443 ssl;
  http2 on;

  server_name gitlab.edotm.net;

  ssl_certificate /path/to/fullchain.pem;
  ssl_certificate_key /path/to/privkey.pem;

  proxy_redirect off;
  
  location / {
    proxy_pass http://gitlab;
    client_max_body_size  10G;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-Proto $scheme;
  }
}

Lines 14-16 are the most crucial in this configuration. Line 14 sets the maximum size of an HTTP request, and it is crucial when using HTTPS as the method for pulling and pushing commits (the default is 1 MB, meaning that a commit cannot be pushed via HTTPS if it’s larger than 1 MB). However, if you plan to use only SSH, I recommend removing it to prevent possible DoS attacks.

Log in to GitLab

Now it’s time to test our GitLab instance. First, we need to retrieve the root account password. To do this, run the command:

docker exec -it gitlab grep "Password:" /etc/gitlab/initial_root_password
lightbulb

Tip

This file will be automatically removed after 24 hours from the creation of the container. If you encounter an error stating that the file cannot be found or still can’t access it, try resetting the root password with the following command:

docker exec -it gitlab gitlab-rake "gitlab:password:reset[root]"

It will prompt you to input a new password and re-enter it to confirm. If this doesn’t work, and you don’t have any critical data stored inside GitLab, you can stop the container, delete all the GitLab data (by deleting the directory or the volume storing it), and restart the container.

Now, you can go to the Admin Area > Users and create a new user.

GitLab Runner in Docker

Now that GitLab is up and possibly behind an NGINX proxy, if you’re running GitLab on the same machine as the one you’ll use for GitLab Runner, it’s recommended not to use NGINX as a proxy for all requests; instead, make the two containers (GitLab and the runner) communicate directly.

This is done in two steps:

Step 1: Add the GitLab custom network

Edit the GitLab Compose file, adding the highlighted lines:

26
27
28
29
30
31
32
33

    networks:
      - nginx-net
      - gitlab

networks:
  nginx-net:
    external: true
  gitlab:

This will create an internal gitlab network to which we will connect the runner.

Step 2: Create the runner instance

Log in to GitLab and go to Admin Area > CI/CD > Runners (/admin/runners). Click on “New instance runner.” Since you probably have GitLab 16.6 or later, you can ignore the warning message. Here, you can add tags to the runner (you can also edit them later), details, and other configurations. When done, click on “Create runner.” You will be given an authentication token (glrt-xxxx...). You will need this later.

Now, create a runner folder and a new docker-compose.yml file with the following configuration:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

version: "3.9"
services:
  runner:
    image: gitlab/gitlab-runner
    restart: always
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./config:/etc/gitlab-runner
    container_name: gitlab-runner-texlive
    networks:
      - gitlab_gitlab
networks:
  gitlab_gitlab:
    external: true

It’s important to connect the runner to the previously created gitlab network. The network’s name won’t be exactly gitlab (unless specified or set up elsewhere, but in that case, you’d know it) but will have the folder name as a prefix. Since I had my GitLab Compose file inside the gitlab/ directory, my network is called gitlab_gitlab.

To set it up, run the command:

docker compose run --rm runner register -u http://gitlab/

You will be prompted to name the runner, set its executor, and input the previously created auth token (the one starting with glrt-).

I recommend setting the executor to docker, but that’s your choice; since you are setting up a runner, I assume you know what you’re doing -- if not, read this answer on Stack Overflow to get the idea.

Once registered, start it up by doing:

docker compose up

If everything goes well, you should not have any error.

lightbulb

Tip

To shut down the runner for now, press CTRL+C.

Step 3: Set up the docker executor

How can we use the Docker executor inside a Docker container? The answer is simple: we will not create Docker containers inside another container; we will make the Runner container able to spawn new containers at the host level, i.e., as if we were spawning them.

How can we do this? It is simple enough: we firstly need to bind the host’s Docker socket to the container’s. And, we already did it: remember line 7 of the Runner Compose file?

7

- /var/run/docker.sock:/var/run/docker.sock

This is crucial, but not sufficient. Since the Docker executor will in turn create other containers, we need to pass the socket inside the other containers that will be created as well. Doing this is simple enough: inside the config/ folder, there is a config.toml file. Let’s add the socket in the volumes field. While we’re at it, let’s set the network_mode to the network gitlab_gitlab created earlier.

The final result should be this:

18
19
20
21
22
23
24
25
26
27
28

  [runners.docker]
    tls_verify = false
    image = "alpine"
    privileged = false
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock"]
    shm_size = 0
    network_mtu = 0
    network_mode = "gitlab_gitlab"
lightbulb

Tip

If you want to easily access the host files from the instances, you can add "/:/host" in the volumes array of the TOML file, so that inside the instances you can access all the host files in /host. Please note that this has security implications and it is recommended to limit the access to the host files.

That wraps up my guide on Dockerizing GitLab behind NGINX and setting up a GitLab Runner. For questions or feedback, drop a comment below. Happy coding!

Tags: Docker Docker Hosting Linux Linux Setup NGINX Nginx GitLab GitLab