Tags: #homelab

Concept: [[Jenkins connection with Git and Container Repository]]

Difficulty: Moderate-Hard

Date: 2025-12-15

Pre-amble
#

Welcome to todays guide. Today we have quite a bit to cover.

The main purpose of this guide is to deploy a full CI/CD pipeline from code to deployment

We will build and run a simple Go + HTML Website docker container.

The website will be a simple petition signing website that uses a document based json database to store information about a petition to be signed against Social Media company with respect to privacy.

Please note that this website is fully constructed with AI and is for demonstration purposes only.

Pre-requisites:
#

Docker setup
Traefik / Reverse Proxy with local domains and SSL setup

Steps:
#

1. Deploy Jenkins
#

Based on https://www.jenkins.io/doc/book/installing/docker/

As everything we have done we will run this service as a docker-compose to make it easier to change and manage.

On your server,

First pull the image:

docker pull jenkins/jenkins:lts-jdk17

Run the docker-compose:

services:
  jenkins:
    image: jenkins/jenkins:lts-jdk17
    container_name: jenkins
    user: root
    networks:
      - jenkins
    ports:
      - "8085:8080"
      - "50000:50000"
    volumes:
      - jenkins-data:/var/jenkins_home
      - /var/run/docker.sock:/var/run/docker.sock
      - /usr/bin/docker:/usr/bin/docker
    restart: unless-stopped

networks:
  jenkins:
    driver: bridge

volumes:
  jenkins-data:

Notes:

/var/run/docker.sock allows Jenkins inside the container to talk to the host Docker daemon. If you dont want this then consider a docker-in-docker approach.
/usr/bin/docker allows the container to run the Docker CLI.
Edit volume path for jenkins-data as required. This defaults will also work
Edit the host ports as you need
I am using my [[traefik-dynamic-editor]] to add this service behind my reverse proxy to get SSL and domain resolution. You can adjust for your setup accordingly.

You can either add the docker compose in [[Portainer]], or create a new file called docker-compose.yml and paste the contents in there and run docker-compose up -d

I have used Portainer to deploy so that i can edit and redeploy the stack as needed

After the container is running
#

Once deployed, check the website http://server-ip:8085, or https://jenkins.domain.io You should be prompted to enter a password

You will see the following in the logs

LF]> Jenkins initial setup is required. An admin user has been created and a password generated.

[LF]> Please use the following password to proceed to installation:

[LF]> 

[LF]> 6a5b74xxxxxxaaabbseeddecb1f4fb41b992997e

[LF]> 

[LF]> This may also be found at: /var/jenkins_home/secrets/initialAdminPassword

We need to use the password to authenticate on the Jenkins website Paste it on the website.

You will then be prompted to install plugins. If you arent exactly sure what you need right now, go for *Install Suggested Plugins

We can always customize later.

The process may take a while till then we can deploy our own custom git provider and container registry.

Once that done more to [Setup Jenkins](##Setting up Jenkins).

2. Deploy GitLab CE
#

See here for what is [[Gitlab CE]]

But in short, it is a github equivalent. For this tutorial i am hosting my own git but you can still use GitHub and follow similar steps. If so skip to [Connecting to Git](#Connecting to Git)

Refer the official installation

GitLab provides both git and container registry so we will hit both birds with one stone.

First pull the gitlab CE image. Its a big image so it will take a while

docker pull gitlab/gitlab-ce:nightly

Then run the service:

services:
  gitlab:
    image: gitlab/gitlab-ce:nightly
    container_name: gitlab
    hostname: gitlab.domain.io
    restart: always
    environment:
      GITLAB_OMNIBUS_CONFIG: |
        # Main GitLab URL (HTTPS via Traefik)
        external_url 'https://gitlab.domain.io'
        
        # Disable GitLab's nginx HTTPS since Traefik handles it
        nginx['listen_port'] = 80
        nginx['listen_https'] = false
        
        # Proxy headers from Traefik
        nginx['proxy_set_headers'] = {
          "Host" => "$$http_host",
          "X-Real-IP" => "$$remote_addr",
          "X-Forwarded-For" => "$$proxy_add_x_forwarded_for",
          "X-Forwarded-Proto" => "https",
          "X-Forwarded-Ssl" => "on"
        }
        
        # SSH Configuration
        gitlab_rails['gitlab_shell_ssh_port'] = 2424
        
        # Container Registry Configuration
        gitlab_rails['registry_enabled'] = true
        registry_external_url 'https://registry.domain.io'
        
        # Registry nginx settings
        registry_nginx['enable'] = true
        registry_nginx['listen_port'] = 5101
        registry_nginx['listen_https'] = false
        
        # Registry proxy headers
        registry_nginx['proxy_set_headers'] = {
          "Host" => "$$http_host",
          "X-Real-IP" => "$$remote_addr",
          "X-Forwarded-For" => "$$proxy_add_x_forwarded_for",
          "X-Forwarded-Proto" => "https",
          "X-Forwarded-Ssl" => "on"
        }
        
        # Registry API URL for internal communication
        gitlab_rails['registry_api_url'] = "http://localhost:5101"
        
        # GitLab Pages Configuration
        pages_external_url 'https://pages.domain.io'
        gitlab_pages['enable'] = true
        
        # Pages nginx settings
        pages_nginx['enable'] = true
        pages_nginx['listen_port'] = 5200
        pages_nginx['listen_https'] = false
        
        # Pages proxy headers
        pages_nginx['proxy_set_headers'] = {
          "Host" => "$$http_host",
          "X-Real-IP" => "$$remote_addr",
          "X-Forwarded-For" => "$$proxy_add_x_forwarded_for",
          "X-Forwarded-Proto" => "https",
          "X-Forwarded-Ssl" => "on"
        }
        
        # Pages settings for external HTTP (required for wildcard domains)
        gitlab_pages['inplace_chroot'] = true
        gitlab_pages['external_http'] = ['0.0.0.0:5201']
        
        # Performance tuning
        postgresql['shared_buffers'] = "256MB"
        postgresql['max_connections'] = 200
        
        # Backup path
        gitlab_rails['backup_path'] = "/var/opt/gitlab/backups"
    ports:
      - '8090:80'
      - '2424:22'
      - '5101:5101'
      - '5201:5201'
    volumes:
      - '/config:/etc/gitlab'
      - '/logs:/var/log/gitlab'
      - '/data:/var/opt/gitlab'
    shm_size: '256m'

    networks:
      - frontend

networks:
  frontend:
    external: true

You can combine this with Jenkins if youd like but ill keep it seperate for ease of changes. Make a note of the port mapping as ive changed the defaults. You may need to configure as per your environment.

Notes:

Set the external url to how Gitlabs can be accessed, either locally http://localhost:8090 or via domain https://gitlab.domain.io

This process can take upto 15mins to launch. Monitor the contaier status to move from starting to running

Setup
#

Meanwhile we can proceed to setup other aspects of the demo

1. Setting up Jenkins
#

![[Pasted image 20251215183319.png]]

![[Pasted image 20251215183520.png]]

Configure your service access URL, Start using Jenkins

Now our jenkins is setup

2. Setting up your Gitlab
#

Open gitlab https://gitlab.domain.io

We need to do two things. First create a user account

Register for new account
Sign up and login
You will be told the admin needs to approve

So now we need to login as admin now,

On your host run

sudo docker exec -it gitlab cat /etc/gitlab/initial_root_password

A long password will be displayed, copy it and login as admin to git lab

username: root
password: { what you copied }

You can now configure to allow user signups but ill leave that on.

Navigate to Settings -> Users -> Your user -> Approve
Logout of root user
Login as normal user

Now you can setup your online repository / project just like you would in github

Create a new private project and set it up first time on the web

(For this demo we can ignore the SSH key error as we will use a PAT)

To create a PAT (Access Token)

Project -> Settings -> Access Tokens -> Add new token
Grant read write access and api access -> Create token
Save the token we will need it again when setting up project

3. Setting up your local project
#

For this example im using the example project Sign petition

Once your project is ready in local editor do the following:

Initiate git repo git init
Allow git to store credential git config --global credential.helper store
Add a new origin git remote add origin http://gitlab.domain.io/<username>/petition-app.git Change as per your use case
Pull remote changes first git pull
Add changes to git git add .
Create first commit git commit -m "First commit"
Set branch git branch -M main
Create a branch to push the code git checkout -b firstcode
Push code git push -u origin firstcode
You will get prompt for username and password. Use your username and the PAT generated earlier
Once pushed, if you check on your gitlab you should see the new branch created.
Create a merge request, and merge the changes
This is the recommended way to preserve git history.
Alternatively you can use SSH to push to code. It is covered in the Notes section [[# Pushing to Git using SSH]]
Once merged, on local system run

git checkout main
git pull origin main

Done!

Alternatively, to skip a few steps you can make your repository unprotected to allow direct push to main branch. This is fine if you are doing just a demo

Wow, if you got till here, you deserve a pat on the back. Youve just hosted your own github and pushed your code there and merged the first merge request.

Take a break. This is a milestone.

But we arent done, now we need to setup Jenkins to read this repo and setup a [[webhooks]] to be able to notify Jenkins to run a docker build.

Connections
#

1. Connect jenkins to Gitlab
#

We can connect jenkins to gitlab in the following way

We will re-use the previous PAT, else a new separate one can be created
Add the token to Jenkins.
Manage Jenkins → Credentials → (Global)
Add new credential:
- Kind: Username with password
- Username: { your-gitlab-username }
- Password: GitLab access token
- ID: gitlab-http-creds
- Description: GitLab HTTPS access token
- Create
Now we can use these credentials when creating the main pipeline and accessing the container registry when needed

2. Connecting Jenkins to Gitlab Container registry
#

In order for Jenkin builds to be pushed to Gitlabs container registry we need to setup Credentials for pushing to the registry.

Recall, registry.domain.io is where our registry is running

We can first test that login is working

docker login http://registry.domain.io

It will ask for username and password. We can use the access key we generated in Gitlab for Jenkins earlier here as well.

Once the login succeeds we know that the registry is setup correctly and can proceed to create the pipeline. (You can try to push a local image to the registry to check. See [[#Pushing to Gitlab Container Registry]])

3. Creating the pipeline
#

Foremost we need to install the Gitlab plugin for webhook support

Manage Jenkins -> Plugins -> Add Plugin - Search Gitlab and install it

We can now create the main docker image pipeline. This step may vary for you based on what your application is and how its architecture is.

For this demo, out application is a simple GO app with a static index.html. I will add linting and testing into the pipeline

Create a Jenkins Pipeline job

Jenkins → New Item

Name: petition-app
Type: Pipeline
OK

Gitlab Connection:

Select the gitlab connection we setup earlier in Connections 1

Triggers:

Select the gitlab trigger to work on PushEvents.
Click on advanced and Scroll to secret Token and generate a new token
Copy the url shown by Jenkins. We have to add this url to GitLab
Gitlab -> Project -> Settings -> Webhooks -> New webhook
Paste the url and the token into the webhook. Keep SSL on
Save and test the webhook
Jenkins should trigger a build

If it fails you may need to allow outbound requests check here

Configure Git repository (HTTPS)

In the job configuration:

Pipeline → Definition

Choose:

Pipeline script from SCM

SCM -> Source control management

SCM: Git
Repository URL: https://gitlab-deep.adgone.co.tz/deep-jiwan/petition-app.git
Credentials:
- Select gitlab-http-creds
Branch: */main
Script Path: Jenkinsfile

This tells Jenkins:

“Clone this repo using HTTPS and this token.”

Jenkinsfile

Your repo will contain a Jenkinsfile.

At a conceptual level it does:

checkout source 
run build steps
 - Get the code
 - Lint the GO code
 - Build the docker image
 - Test the docker image
 - Push the image to continer registry
 - Cleanup Jenkins workspace

The following is my build pipeline for the demo application. You can customize as per your requirements. Make sure to make the edits are per your domains

pipeline {
  agent any
  
  environment {
    IMAGE = "registry.domain.io/<username>/petition-app/petition-app"
    TAG = "build_${BUILD_NUMBER}"
    TEST_CONTAINER = "test-${BUILD_NUMBER}"
    TEST_PORT = "8089"
    SERVER = "192.168.88.248"
  }
  
  stages {
    stage('Checkout') {
      steps {
        checkout scm
      }
    }
    
    stage('Lint') {
      agent {
        docker {
          image 'golangci/golangci-lint:v1.59.0'
          reuseNode true
        }
      }
      steps {
        sh 'go mod download'
        sh 'golangci-lint run --timeout=3m --fast ./...'
        sh 'echo "Lint Passed"'
      }
    }
    
    stage('Build') {
      steps {
        sh "docker build -t ${IMAGE}:${TAG} -t ${IMAGE}:latest ."
        sh 'echo "Build Completed"'
      }
    }
    
    stage('Test') {
      steps {
        script {
          sh "docker run -d --name ${TEST_CONTAINER} -p ${TEST_PORT}:8080 ${IMAGE}:${TAG}"
          sleep 5
          sh """
            curl -f http://${SERVER}:${TEST_PORT}/health || \
            curl -f http://${SERVER}:${TEST_PORT}/ || \
            (docker logs ${TEST_CONTAINER} && exit 1)
          """
          sh 'echo "Tests Passed"'
        }
      }
    }
    
    stage('Push') {
      steps {
        withCredentials([usernamePassword(
          credentialsId: 'gitlab-http-creds',
          usernameVariable: 'USER',
          passwordVariable: 'PASS'
        )]) {
          sh 'echo "$PASS" | docker login registry.domain.io -u "$USER" --password-stdin'
          sh "docker push ${IMAGE}:${TAG}"
          sh "docker push ${IMAGE}:latest"
          sh 'echo "Push Completed"'
        }
      }
    }
  }
  
  post {
    always {
      sh "docker stop ${TEST_CONTAINER} || true"
      sh "docker rm ${TEST_CONTAINER} || true"
      // sh "docker rmi ${IMAGE}:${TAG} ${IMAGE}:latest || true"
      sh 'ehco "Cleaned up Docker resources"'
      cleanWs()
    }
  }
}

This should now work once you push to GitLab.

Check the Jenkins builds, a new build should appear that will follow the above stages.

4. Success
#

Your app should now build and follow the stages defined in the Jenkinsfile.

For me this was the output:

![[Pasted image 20251222153113.png]] Jenkins pipeline

![[Pasted image 20251222150831.png]] Gitlab Container Registry (Petition-app -> Deploy -> Container Registry).

![[Pasted image 20251222151104.png]] Final Application output

Notes
#

This section contains small guides to different alternate aspects of this demo

Pushing to Gitlab Container Registry
#

docker login registry-deep.adgone.co.tz

Provide username
Provide Access token as password

Build Image

docker build -t registry.domain.io/<username>/petition-app .

OR Tag and existing image to push

docker tag petition-app:latest registry.domain.io/<username>/petition-app

Push to the Registry

docker push registry.domain.io/<username>/petition-app

Done!

Pushing to Git using SSH
#

Author

Deep Jiwan

Building hacky solutions that save time and make my life easier. Not too sure about yours :)

Pre-amble#

Pre-requisites:#

Steps:#

1. Deploy Jenkins#

After the container is running#

2. Deploy GitLab CE#

Setup#

1. Setting up Jenkins#

2. Setting up your Gitlab#

3. Setting up your local project#

Connections#

1. Connect jenkins to Gitlab#

2. Connecting Jenkins to Gitlab Container registry#

3. Creating the pipeline#

4. Success#

Notes#

Pushing to Gitlab Container Registry#

Pushing to Git using SSH#