How to create a ci cd pipeline for building docker images

This post walks through how to create a CI / CD pipeline for building docker images.
We will create the docker image from a docker file. The docker file will be stored in a GitLab repository. In the GitLab repository we will create a gitlab-ci.yml file which will define our CI/CD pipeline to build and push our docker image to docker hub. This pipeline will also be triggered when we push changes to the code in our repository.
Prerequisites
You will need a GitLab account and a Docker hub account before you can follow along with the steps below.
Follow our post here to get set up with a GitLab account. Go to the Docker hub site here to create a docker account.
Steps
We will start by creating a project in GitLab. When logged in to GitLab click on New Project.
Fill in the project details and click the Initialize repository with a README option.
Once your project is created create a new branch for you project. We will do our initial development work on this branch and merge to master when we are happy our code is good enough to be used in production.
To create a new branch, within your project go to Repository then Branches.
Then click on new branch.
Give your new branch a name and click on Create branch.
Now on our workstation we are going to start creating our dockerfile.
I am working on an Ubuntu 18.04 workstation. Steps may differ if you are on a different OS.
In your GitLab project click on clone and copy either the HTTPS or SSH details.
Clone your repository from your new GitLab project to your workstation, cd into the new directory and open vscode or whatever text editor you use.
Create a new dockerfile in the root of your directory.
My docker file looks like this. It is a docker image used for doing IaC work and includes an install of ansible, packer, terraform and gcloud.
When you have your docker file finalised, run docker build locally to test it.
docker build .
If your image builds successfully then you should see the following message and you know your docker file is good.
You could also login to your docker image locally to test if it is good.
First you would need to list your docker images with
docker images ls
Note your docker image id and run
docker run -it image id
Now that your docker file is ready, you should create a .gitlab-ci file.
This file will define our CI/CD pipeline to build and push our docker image to docker hub.
In the root of your project directory create a file called .gitlab-ci.yml
My gitlab.ci file looks like the following
Within the file is a reference to a GitLab secret variable called DOCKERHUBPW
. This variable will store your docker hub password. You will need to create this secret variable within your GitLab project. To do this, within your project go to Settings > CI / CD
In here under variables, input DOCKERHUBPW as the key and add your docker hub password as the value.
Once done click Save variables.
At this point you should have your docker file, gitlab-ci.yml file and your secret variable in place. If you do you are good to git push your code to your dev branch.
To push your code do a
git add .
git commit
git push
When you push your code a pipeline should run in GitLab. What happens in this pipeline is based on what is defined in your gitlab-ci.yml file. If your pipeline file is similar to mine then your pipeline should build your docker image, push it to docker hub then pull the image and test it.
To see your pipeline running go to your GitHub project and on the left go to CI / CD and then pipelines. You should see something like the following.
Click in to the pipeline to see the jobs in the pipeline.
Hopefully your jobs will run successfully. If they do you should have a new docker image in your docker hub account.
If your pipeline file is the same as mine and you click into your image, you will notice that the image has the tag dev.
The image is tagged with dev because this is what we have specified in our gitlab-ci.yml file. We told the pipeline if this image is being created from a branch that is not master then tag the image with the tag dev. The except: - master code is what defines this.
Now that we are happy with our code and we have tested our image we can merge our dev branch to master. When we do that a new pipeline will run and create our docker image with the tag prd. This prd image is the image you would then use in production.
To merge your code using GitLab you need to go into your project and on the left click on merge requests on the left.
Click new merge request.
Select your dev branch as the source branch and master as the target.
Click Compare branches and continue.
Give your MR a title
Scroll down and click Submit merge request.
Your MR is now ready for review. You can review changes under the changes tab. If you are happy you can click the merge button.
Once you click Merge you pipeline should start to run. Click into your pipelines to view the running pipeline.
Once your pipeline completes successfully you should have a container in docker hub with the tag prd ready for use in production.
That's it, you now have a pipeline for creating docker images.
If you need to make a change to your docker image you should create a feature branch, make your changes and push the changes to your GitLab repo. Your pipeline will then run and update your dev image. Test your dev image and when your are happy, merge your feature branch to master. When your feature branch merges your pipeline will run again and update your prd image.
Leave a comment below if this worked for you or not.
What if you have two active feature branches going on to make two different change sets. How do you know the pipeline is pulling the right image if both feature branches happen to build at the same time?