Improving the execution time of CI pipelines

Executing a large number of tests, especially integration tests, takes a lot of time. For instance, the pipeline of one of our projects for each Pull Request previously took nearly 30 minutes, including over 1 thousand test cases. This article guides you through several good techniques that we have discovered and applied to improve the time-consuming process.

Parallel stages

Analyze the current phases in your pipeline and categorize them in parallel. For example, we can separate the build and verify code of Node.js and Maven modules simultaneously in our Jenkins pipelines. Please mind using the setting failFast whether you want to abort the pipeline immediately.

Parallel test execution

If you use Maven, the plugin maven-failsafe-plugin is used to execute integration tests during phases integration-test and verify the build lifecycle. It allows us to execute tests in parallel. There are many settings related to parallel test execution, so make sure to configure them correctly for your environment. For example, we can execute tests in parallel using the classes mode and set threadCount=7 for the Jenkins Agent with 8 CPU cores.

Application profiles

There are certain integration tests that don't require the use of a database. These tests focus on scenarios like error handling, authentication filters, or loading configuration. For example, initializing the database with Testcontainers is time-consuming. If you use Spring Boot, you can utilize a different profile and use the H2 in-memory database for these tests. The test class will be annotated with the @ActiveProfiles annotation to run with the specified profile.

Comments

The HelloWorld example of JSF 2.2 with Myfaces

I just did by myself create a very simple app "HelloWorld" of JSF 2.2 with a concrete implementation Myfaces that we can use it later on for our further JSF trying out. I attached the source code link at the end part. Just follow these steps below: 1. Create a Maven project in Eclipse (Kepler) with a simple Java web application archetype "maven-archetype-webapp". Maven should be the best choice for managing the dependencies , so far. JSF is a web framework that is the reason why I chose the mentioned archetype for my example. 2. Import dependencies for JSF implementation - Myfaces (v2.2.10) into file pom.xml . The following code that is easy to find from http://mvnrepository.com/ with key words "myfaces". <dependency> <groupId>org.apache.myfaces.core</groupId> <artifactId>myfaces-api</artifactId> <version>2.2.10</version> </dependency> <dependency> <groupId>org.apache.myfaces.core<...

Junit - Test fails on French or German string assertion

In my previous post about building a regex to check a text without special characters but allow German and French . I met a problem that the unit test works fine on my machine using Eclipse, but it was fail when running on Jenkins' build job. Here is my test: @Test public void shouldAllowFrenchAndGermanCharacters(){ String source = "ÄäÖöÜüß áÁàÀâÂéÉèÈêÊîÎçÇ"; assertFalse(SpecialCharactersUtils.isExistSpecialCharater(source)); } Production code: public static boolean isExistNotAllowedCharacters(String source){ Pattern regex = Pattern.compile("^[a-zA-Z_0-9_ÄäÖöÜüß áÁàÀâÂéÉèÈêÊîÎçÇ]*$"); Matcher matcher = regex.matcher(source); return !matcher.matches(); } The result likes the following: Failed tests: SpecialCharactersUtilsTest.shouldAllowFrenchAndGermanCharacters:32 null A guy from stackoverflow.com says: "This is probably due to the default encoding used for your Java source files. The ö in the string literal in the J...

Why Business Rules Engine Matter

"A good DSL minimizes the 'communication gap' between a domain concept and the code that implements it" - Robert C. Martin A Use Case It looks like adopting a new technology is always driven by a business need. The organizations, such as banks, have their own business processes (such as data gathering and document management). These processes are different from others but usually the differences are not big. How do we build a product which can be reused the similar features but still be adaptable with the specific requirements of each bank? Then, the answer is we should have a library/framework. There is an idea that we can implement this library/framework by using a business rules engine. Which has the following benefits: Be able to modify implementation of business domain at runtime (such as XML, CSV) S ource code is more readable for both developers and domain experts. Therefore, source code can be consider it as a document Since Business Rules Engine...

Java 8 - Persistent data structure

The following is a series of posts about "functional programming in Java" which is the result of my understanding by reading the book " Java 8 in Action: Lambdas, Streams, and Functional-style Programming, by Alan Mycroft and Mario Fusco ". 1. Why functional programming? 2. Functional programming in Java 8 3. Java 8 - Using Functions as Values 4. Java 8 - Persistent data structure Persistent data structure is also known as a simple technique but it's very important. Its other names are functional data structure and immutable data structure. Why is it "persistent"? Their values persist and are isolated from changes happening elsewhere. That's it! This technique is described as below: If you need a data structure to represent the result of a computation, you should make a new one and not mutable an existing data structure. Destructive updates version public static A doSomething(A a){ a.setProp1("new value"); return...

Building Axon.ivy Projects on Bitbucket Pipelines

Read me if you don't know what Axon.ivy (Ivy) is. Motivation - Ivy projects are designed to be built on a continuous integration (CI) server like Jenkins - Today, Bitbucket supports for CI with Bitbucket Pipelines - We're using Bitbucket. Then, why not? It must be very cool and convenient for us if we can centralize our CI and VCS (version control system) tools in one place. Here is an approach We have to use a maven plugin called project-build-plugin to build ivy projects. This plugin requires an instance of Ivy engine during building time. Bitbucket Pipelines allows us to specify our own docker image as a build environment. What we need to do is to prepare our docker image with needed stuffs such as JDK, Maven, Ivy engine, etc. Step 1. Prepare Docker images For testing purpose, I already created two docker images: Maven and Axon.ivy engine. They are now available on Docker Hub This image for Maven using Oracle JDK 8 This image for Axon.iv...

vhandit

Search This Blog