Perfect CI-CD with Jenkins pipeline on Google Kubernetes engine

Articles about Jenkins pipeline

Why we work with Jenkins Pipeline? Around 50 developers work at Ackee to develop, test and deliver apps for our demanding customers. We use several methods for this. With multiple git pushes and merge requests per hour, we need a fast and optimized process. The CI/CD using multiple methods and clouds respectively. Automating other deployment goals is very important to us. Therefore, we use the powerful Jenkins pipeline with the shared pipeline library together with Jenkins and Gitlab.

Our backend is mainly based on Kubernetes cluster as well as NodeJS, MongoDB, MySQL and some PHP containers. The frontend with React, Middleman and other methods must be quickly tested and deployed to the appropriate web servers. This can be Google Storage Buckets or FTP web hosting – depending on how the customer wants it to be.

The Android and iOS teams also need a fast and reliable CI/CD . Generally all teams need a way to build and test merge requests. In addition, there is a constant need to automate something in the infrastructure. That's why the scripted automation platform for the DevOps team is a high priority for us.

Jenkins Pipeline is the better alternative

Some time ago we used Jenkins Freestyle Jobs to try out a repository. We built it into the console with simple bash snippets and shared it with other servers using SSH, Rsync, etc. Made available.

Then came Jenkins Pipeline. Jenkins Pipeline is a collection of plugins that support the inclusion and integration of always-on delivery pipelines to Jenkins. So an alternative to the old freestyle jobs.
With Jenkins Pipeline, all you have to do is create a pipeline or a multibranch pipeline job and define the pipeline as code . The code can be part of the job definition or directly included as a Jenkins file in the project repository.

What Jenkins Pipeline does

The Jenkins pipeline has different levels, which are visible thanks to the Visualizer plugin. What the Jenkins pipeline can do is this:

– She builds the Node App with NPM ("sh" means Shell Call ),
– It tests the whole thing
– It builds a Docker image and manages it
– It cleans up the workspace and
– it notifies Slack about the pipeline result.

Slacknotify is a Slack plugin call . This means that plugins must support Jenkins Pipeline syntax to be considered a helpful function with a choice of parameters.

jenkins pipeline

Jenkins Pipeline is extremely powerful. It does more or less everything you want it to do. Since it uses the Groovy language, you can use Java libraries in the pipeline.

Jenkins Shared Pipeline Library

Since we work on many projects at the same time, we would have a lot of similar code or the exact same Jenkinsfile in all NodeJS and other repositories.

Imagine that a simple change in the slackNotify function call syntax or any other CIi client syntax would mean that we need to change the pipeline code in all repositories and branches!
By the way, this happened the other day with GCloud cli API client syntax . The old syntax was rejected and changed GCloud Docker Pus h to GCloud Docker – push. In our case we don't want to find the pipeline logic in the project repository. The only thing we want to have there is the pipeline configuration. This can be easily achieved by using Jenkins Shared Pipeline Library. This is a Groovy Git repository with the following folder structure.

This shared pipeline repository is checked every time a pipeline job is started. You can put it in your Jenkinsfile and use it there. You can even define branches.

Or you can load it implicitly . This can be set in the Jenkins configuration settings.

Configuration of Jenkinsfile

If you use the shared pipeline library, the jenkinsfile in the project repository can look like this:

Example of a Jenkins pipeline

The Jenkinsfile in the previous section calls a function PipelineNodeJS. This is a global function that is in the shared pipeline library in vars/PipelineNodeJS.Groovy is defined.

Merge Request Builder

For the merge request flow with Gitlab, we script a Jenkins pipeline job that uses the Gitlab integration plugin.
Here comes the merge request builder job example for iOS:

Use GitLabBuild and GitlabCommitStatus to visualize the Jenkins pipeline in Gitlab and allow the MR to merge into Gitlab respectively. To disallow. This job can be used globally for all iOS merge requests. Set the WebHook for this Jenkins job from your repository – and you're done.

An annoying but important step is the SCM checkout , where source and target branches are merged together. The ENV data (env.Gitlabsourcebranch and env.Gitlabtargetbranch) are parsed from the Gitlab webhook.
You can easily set up the rebuilding of the merge request again if a new push to source or target branch has been done.

Another cool feature is that you can rerun the Jenkins pipeline by commenting in MR. We use the phrase "rebuild pls" and Jenkins automatically rebuilds the job!

Job DSL and Seed Jobs

Thanks to the Job DSL plugin, we can dynamically generate Jenkins pipeline jobs via the Jenkins DSL API.
A job that generates other jobs is called a seed job.
Here comes an example of generating an iOS merge request job:

Test the production pipeline

Now comes the moment when we need to build, test and ship our Jenkins pipeline. CI/CD for the CI/CD Pipeline.

Yo dawg, I heard you like Inception.

We place a simple Jenkins file in the shared pipeline library and use it to "seed" the jobs in the previous chapter.

Leave a Reply

Your email address will not be published. Required fields are marked *