Improving the execution time of CI pipelines

Executing a large number of tests, especially integration tests, takes a lot of time. For instance, the pipeline of one of our projects for each Pull Request previously took nearly 30 minutes, including over 1 thousand test cases. This article guides you through several good techniques that we have discovered and applied to improve the time-consuming process.

Parallel stages

Analyze the current phases in your pipeline and categorize them in parallel. For example, we can separate the build and verify code of Node.js and Maven modules simultaneously in our Jenkins pipelines. Please mind using the setting failFast whether you want to abort the pipeline immediately.

Parallel test execution

If you use Maven, the plugin maven-failsafe-plugin is used to execute integration tests during phases integration-test and verify the build lifecycle. It allows us to execute tests in parallel. There are many settings related to parallel test execution, so make sure to configure them correctly for your environment. For example, we can execute tests in parallel using the classes mode and set threadCount=7 for the Jenkins Agent with 8 CPU cores.

Application profiles

There are certain integration tests that don't require the use of a database. These tests focus on scenarios like error handling, authentication filters, or loading configuration. For example, initializing the database with Testcontainers is time-consuming. If you use Spring Boot, you can utilize a different profile and use the H2 in-memory database for these tests. The test class will be annotated with the @ActiveProfiles annotation to run with the specified profile.

Comments

Sharing a virtualenv across several Python projects using Pipenv

There is a standard library for all projects in Python. However, several projects don’t always have the same dependencies all the time. That is where virtual environments come to play. You can follow this official document to use two separated tools virtualenv and pip to fulfill that need. My preferred alternative is to use pipenv . Pipenv is easy to use and convenient. The following are my steps to make a shared virtualenv for my all projects which requires the same dependencies. Step 1. Create an isolated virtualenv. python -m venv my-shared-env Step 2. Create a symbolic link to the created virtualenv. cd project_1 ln -s ~/.local/share/virtualenvs/my-shared-env .venv I have encountered the following issue at step 1. FileNotFoundError: [Errno 2] No such file or directory: '{my_project_path}/.venv/bin/pip': '{my_project_path}/.venv/bin/pip' The root cause was I tried to create virtualenv by running pipenv install and renaming the generated virtualenv to ...

Junit - Test fails on French or German string assertion

In my previous post about building a regex to check a text without special characters but allow German and French . I met a problem that the unit test works fine on my machine using Eclipse, but it was fail when running on Jenkins' build job. Here is my test: @Test public void shouldAllowFrenchAndGermanCharacters(){ String source = "ÄäÖöÜüß áÁàÀâÂéÉèÈêÊîÎçÇ"; assertFalse(SpecialCharactersUtils.isExistSpecialCharater(source)); } Production code: public static boolean isExistNotAllowedCharacters(String source){ Pattern regex = Pattern.compile("^[a-zA-Z_0-9_ÄäÖöÜüß áÁàÀâÂéÉèÈêÊîÎçÇ]*$"); Matcher matcher = regex.matcher(source); return !matcher.matches(); } The result likes the following: Failed tests: SpecialCharactersUtilsTest.shouldAllowFrenchAndGermanCharacters:32 null A guy from stackoverflow.com says: "This is probably due to the default encoding used for your Java source files. The ö in the string literal in the J...

Strategy Design Pattern

For example, I have a program with an Animal abstract class and two sub-classes Dog and Bird. I want to add new behavior for the class Animal, this is "fly". Now, I face two approaches to solve this issue: 1. Adding an abstract method "fly" into the class Animal. Then, I force the sub-classes should be implemented this method, something like: public abstract class Animal{ //bla bla public abstract void fly(); } public class Bird extends Animal{ //bla bla public void fly(){ System.out.println("Fly high"); } } public class Dog extends Animal{ //bla bla public void fly(){ System.out.println("Cant fly"); } } 2. Creating an interface with method "fly" inside. The same issue to an abstract class, I force the classes these implement this interface should have a method "fly" inside: public interface Flyable{ public void fly(); } public class Bird implements Flyable{ //bla bla public void fly(){ System.out.pr...

The HelloWorld example of JSF 2.2 with Myfaces

I just did by myself create a very simple app "HelloWorld" of JSF 2.2 with a concrete implementation Myfaces that we can use it later on for our further JSF trying out. I attached the source code link at the end part. Just follow these steps below: 1. Create a Maven project in Eclipse (Kepler) with a simple Java web application archetype "maven-archetype-webapp". Maven should be the best choice for managing the dependencies , so far. JSF is a web framework that is the reason why I chose the mentioned archetype for my example. 2. Import dependencies for JSF implementation - Myfaces (v2.2.10) into file pom.xml . The following code that is easy to find from http://mvnrepository.com/ with key words "myfaces". <dependency> <groupId>org.apache.myfaces.core</groupId> <artifactId>myfaces-api</artifactId> <version>2.2.10</version> </dependency> <dependency> <groupId>org.apache.myfaces.core<...

Styling Sort Icons Using Font Awesome for Primefaces' Data Table

So far, Primefaces has used image sprites for displaying the sort icons. This leads to a problem if we want to make a different style for these icons; for example, I would make the icon "arrow up" more blurry at the first time the table loading because I want to highlight the icon "arrow down". I found a way that I can replace these icons with Font Awesome icons. We will use "CSS Pseudo-classes" to achieve it. The hardest thing here is that we should handle displaying icons in different cases. There is a case both "arrow up" and "arrow down" showing and other case is only one of these icons is shown. .ui-sortable-column-icon.ui-icon.ui-icon-carat-2-n-s { background-image: none; margin-left: 5px; font-size: 1.1666em; position: relative; } .ui-sortable-column-icon.ui-icon.ui-icon-carat-2-n-s:not(.ui-icon-triangle-1-s)::before { content: "\f106"; font-family: "FontAwesome"; position: ...

vhandit

Search This Blog