Improving the execution time of CI pipelines

Executing a large number of tests, especially integration tests, takes a lot of time. For instance, the pipeline of one of our projects for each Pull Request previously took nearly 30 minutes, including over 1 thousand test cases. This article guides you through several good techniques that we have discovered and applied to improve the time-consuming process.

Parallel stages

Analyze the current phases in your pipeline and categorize them in parallel. For example, we can separate the build and verify code of Node.js and Maven modules simultaneously in our Jenkins pipelines. Please mind using the setting failFast whether you want to abort the pipeline immediately.

Parallel test execution

If you use Maven, the plugin maven-failsafe-plugin is used to execute integration tests during phases integration-test and verify the build lifecycle. It allows us to execute tests in parallel. There are many settings related to parallel test execution, so make sure to configure them correctly for your environment. For example, we can execute tests in parallel using the classes mode and set threadCount=7 for the Jenkins Agent with 8 CPU cores.

Application profiles

There are certain integration tests that don't require the use of a database. These tests focus on scenarios like error handling, authentication filters, or loading configuration. For example, initializing the database with Testcontainers is time-consuming. If you use Spring Boot, you can utilize a different profile and use the H2 in-memory database for these tests. The test class will be annotated with the @ActiveProfiles annotation to run with the specified profile.

Comments

My must-have apps for daily work

There is no doubt that cool apps can help us be more productive and enjoyable at work. For the time being, I really love the following apps which are used by me almost every day. 1. A personal Kanban In fact, a personal kanban is the most useful app for me. Why does it matter? It is not just a to-do list, but it keeps me motivated every day because it helps me be able to know what my "big picture" is. I usually set up my plans together with a path to reach them. KanbanFlow is my preferred tool. KanbanFlow 2. A terminal Needless to say, a terminal is a must-have app for every developer, especially the ones use macOS/Linux. Due to its importance, I love to decorate and enhance it to be super exciting with various tools such as iTerm , oh-my- zsh , and thefuck . ;) iTerm + oh-my-zsh 3. A documentation "ecosystem" As a developer, I can not remember all things that I have experimented a day. Moreover, a document is really useful for sharing an...

Regex - Check a text without special characters but German, French

Special characters such as square brackets ([ ]) can cause an exception " java.util.regex.PatternSyntaxException " or something like this if we don't handle them correctly. I had met this issue. In my case, my customers want our application should allow some characters in German and French even not allow some special characters. The solution is that we limit the allowed characters by showing the validation message on GUI. For an instance, the message looks like the following: "This field can't contain any special characters; only letters, numbers, underscores (_), spaces and single quotes (') are allowed." I used Regular Expression to check it. For entering Germany and French, I actually don't have this type of keyboard, so I referred these sites: * German characters: http://german.typeit.org/ * French characters: http://french.typeit.org/ Here is my code: package vn.nvanhuong.practice; import java.util.regex.Matcher; import java.util...

Changing source code at run-time with Service Locator pattern

I have a service to get some data but the result can be different basing on where the implementation is. Technically, I have two or more concrete implementation of an interface and I am able to switch using these concrete classes at run-time. That means I have a place to configure it without re-deploying the application. In order to overcome this issue, I use Service Locator design pattern and here I only care about two advantages below: Encapsulating the specific implementation, we just declare the name and don't care about the implementation of the service. Changing the implementation at run-time. Client : an object that invokes the services via Service Locator Business services : services that is used by Client.

My 2017 Review

Passion for System Design After finishing a one year project, my longest stable team (lasted for 3 years) was separated into two smaller teams. Sadly, but that was a good chance for me to become a key member in my new team. My preferred skills were about system architectures; therefore most of the tasks of building the application structures were handled by me. In order to enhance my design system skills, I have spent much my time for reading books closely after work. These following books help me a lot. - Object-Oriented Thought Process | Matt Weisfeld - Head First Design Pattern | Elisabeth Freeman and Kathy Sierra - Java 8 in Action: Lambdas, Streams, and Functional-style Programming | Alan Mycroft and Mario Fusco Junior Technical Architect I was requested to join a technical architect team (aka Team. Alpha) where I actually had gained experiences almost on interviewing candidates for my company (lol). Besides, I noticed myself must improve the skills of convinci...

The culture war at the heart of open source

I enjoyed reading this post: https://words.steveklabnik.com/the-culture-war-at-the-heart-of-open-source Here is my highlight: If you ask a random developer what “open source” means to them, you won’t often hear “software that follows the Open Source Definition.” If you ask them “what’s the difference between free software and open source software,” you’ll often hear “aren’t those the same thing?” or “you can charge money for open source software, it’s not always free.” You may even hear “it’s on GitHub.” In brief, there was a man Richard Stallman (and his team) starting a project called "GNU Project" in 1983. This project was about to develop a free software system. The term "free" here meant "freedom", not only about the price. The Free Software Foundation appeared to support this project. But then, the term "free software" was so ambiguous. "In addition, the ambiguity of the term “free software” was seen as discouraging busine...

vhandit

Search This Blog