Script (snippet) to automate build image from git and deploy to Kubernetes

Background

When my company start using Kubernetes, I start working on all image building and deployment activities, mainly with command line tools from docker as well as kubectl.

When teams size go up, it start getting more effort needed to be the single deployment person. So some kind of automation is needed.

How I did it before this script

As I use Kustomize to perform multiple environments (development , production) deployment, so mainly I prepared the yaml file so that I can do it by running the following:

kubectl kustomize <env folder path> | kubectl apply -f -

And the folder structure as follow:

The beauty of the above, is when the config files are not modified, the corresponding items is not re-configured in the Kubernetes cluster, so I can run the same again and again.

What is needed to do manually before running this script if there is code change (assuming no configuration change)

1. Build the local container image
2. Tag the image to prepare to push to remote docker container registry
3. Push the image to remote registry
4. Update the deployment yaml files for image version
5. (Finally run the kubectl command stated above)

What the script’s objective (and what it does not cover)

As the major code change does not require Kubernetes configuration changes (like ingress, config map…), so I would want the script to only cater on change of image content.

Key steps of the scripts

1. Check git ssh identity added
2. Check necessary software installation (e.g. docker, kubectl)
3. Login docker to remote docker container registry
4. Check if accessing correct Kubernetes cluster
5. Pull a list of git remote tag, allow user to choose which tag to be deployed
6. Clone the code of particular git tag selected
7. Build the image
8. Tag the image for pushing to remote docker container registry
9. Push the image
10. Update the Kubernetes image tag
11. Clean up the cloned git directory finally

1. Check git ssh identity added

In order to run git command on remote repository, we would make sure to follow the document to create ssh identity and add the identity in the terminal (shell).

My checking could be as following:

The command “git remote show origin” would throw error if user not added the ssh identity or not having permission, and so the resulting GIT_REPO would be empty and the script would exit

2. Check necessary software installation (e.g. docker, kubectl)

We can use “which” to check if the software/tool is installed (on the shell’s search path)

3. Login docker to remote docker container registry

This is a very insecure setup, as the user name, password is stored in environment variable.

4. Check if accessing correct Kubernetes cluster

the idea is run the kubectl cluster-info, check the output to see if the message output match the expected IP.

The expected output of “kubectl cluster info” is as following:

The above script would return an array if it match, the better way is probably take the first line for comparison (haven’t test it yet):

kubectl cluster-info | head -n 1 | grep -Po $EXPECTED_KUBE_IP

5. Pull a list of git remote tag, allow user to choose which tag to be deployed

This portion of script would try pull a list of latest git order by creation date, and take the last 5 items

The “git ls-remote --tags --sort creatordate -q | tail -n 5" would result in following:

Then we use the awk command:

awk ‘{split($2,tag,”/”); print tag[3]}’

This command would split the 2nd column by “/” and put it into “tag” variable, in which the 3rd part of this tag is the git tag we need.

Additionally, in my case I have control on allowing to push only dev_xxxx and prod_xxxx tag name.

6. Clone the code of particular git tag selected

We could clone the repo, but only take depth of 1 of that particular tag so we would not be downloading the whole git history,

7–10. Build image, tag, push and set image with kubectl

The following commands do the tasks respectively.

Note that the kubectl set image would require a filter to select the particular deployment item to update, and my deployment is like follow for this case:

Conclusion

I like this script for now, as it allow my teammates to do the deployment once they push the code into git and tag it (except some environment change is needed, like ingress or config map)

The ultimate goal is to have a CD/CI solution that can allow more flexibility and complexity.

I did explored Jenkins X before I build this script…I do not particularly understand it, an I believe Argo CD is something I would explore next but I am still very confused it Argo CD cover the image building and push part of the deployment.