Improving the execution time of CI pipelines

Executing a large number of tests, especially integration tests, takes a lot of time. For instance, the pipeline of one of our projects for each Pull Request previously took nearly 30 minutes, including over 1 thousand test cases. This article guides you through several good techniques that we have discovered and applied to improve the time-consuming process.

Parallel stages

Analyze the current phases in your pipeline and categorize them in parallel. For example, we can separate the build and verify code of Node.js and Maven modules simultaneously in our Jenkins pipelines. Please mind using the setting failFast whether you want to abort the pipeline immediately.

Parallel test execution

If you use Maven, the plugin maven-failsafe-plugin is used to execute integration tests during phases integration-test and verify the build lifecycle. It allows us to execute tests in parallel. There are many settings related to parallel test execution, so make sure to configure them correctly for your environment. For example, we can execute tests in parallel using the classes mode and set threadCount=7 for the Jenkins Agent with 8 CPU cores.

Application profiles

There are certain integration tests that don't require the use of a database. These tests focus on scenarios like error handling, authentication filters, or loading configuration. For example, initializing the database with Testcontainers is time-consuming. If you use Spring Boot, you can utilize a different profile and use the H2 in-memory database for these tests. The test class will be annotated with the @ActiveProfiles annotation to run with the specified profile.

Comments

The HelloWorld example of JSF 2.2 with Myfaces

I just did by myself create a very simple app "HelloWorld" of JSF 2.2 with a concrete implementation Myfaces that we can use it later on for our further JSF trying out. I attached the source code link at the end part. Just follow these steps below: 1. Create a Maven project in Eclipse (Kepler) with a simple Java web application archetype "maven-archetype-webapp". Maven should be the best choice for managing the dependencies , so far. JSF is a web framework that is the reason why I chose the mentioned archetype for my example. 2. Import dependencies for JSF implementation - Myfaces (v2.2.10) into file pom.xml . The following code that is easy to find from http://mvnrepository.com/ with key words "myfaces". <dependency> <groupId>org.apache.myfaces.core</groupId> <artifactId>myfaces-api</artifactId> <version>2.2.10</version> </dependency> <dependency> <groupId>org.apache.myfaces.core<...

Coding Exercise, Episode 1

I have received the following exercise from an interviewer, he didn't give the name of the problem. Honestly, I have no idea how to solve this problem even I have tried to read it three times before. Since I used to be a person who always tells myself "I am not the one good at algorithms", but giving up something too soon which I feel that I didn't spend enough effort to overcome is not my way. Then, I have sticked on it for 24 hours. According to the given image on the problem, I tried to get more clues by searching. Thanks to Google, I found a similar problem on Hackerrank (attached link below). My target here was trying my best to just understand the problem and was trying to solve it accordingly by the Editorial on Hackerrank. Due to this circumstance, it turns me to love solving algorithms from now on (laugh). Check it out! Problem You are given a very organized square of size N (1-based index) and a list of S commands The i th command will follow t...

MS SQL Server Views

"Creates a virtual table whose contents (columns and rows) are defined by a query. Use this statement to create a view of the data in one or more tables in the database. For example, a view can be used for the following purposes: - To focus, simplify, and customize the perception each user has of the database. - As a security mechanism by allowing users to access data through the view, without granting the users permissions to directly access the underlying base tables. - To provide a backward compatible interface to emulate a table whose schema has changed." [1] Beside that, our team used view in order to improve the performance of our web apps when a database has a very complicated relationship between its tables by using ORM Frameworks such as Hibernate. Example code: --create CREATE VIEW placeholders AS SELECT EMPKEY AS empkey, CONNUMB AS connumb, EMPNBR AS empNbr, ACEEMPN AS empFirstName, ACEEMPFN AS empLastName, EMPNAM AS empFullName, ...

How to convert time between timezone in Java, Primefaces?

I use the calendar Primefaces component with timeOnly and timeZone attributes for using only hour format (HH:mm). Like this: <p:calendar id="xabsOvertimeTimeFrom" pattern="HH:mm" timeOnly="true" value="#{data.dateFrom}" timeZone="#{data.timeZone}"/> We can convert the value of #{data.dateFrom} from GMT/UTC time zone to local, conversely, from local time zone to GMT/UTC time zone. Here is my functions: package vn.nvanhuong.timezoneconverter; import java.text.ParseException; import java.text.SimpleDateFormat; import java.util.Calendar; import java.util.Date; import java.util.TimeZone; public class TimeZoneConverter { /** * convert a date with hour format (HH:mm) from local time zone to UTC time zone */ public static Date convertHourToUTCTimeZone(Date inputDate) throws ParseException { if(inputDate == null){ return null; } Calendar calendar = Calendar.getInstance(); calendar.setTime(inputDate); int ...

Managing JAR files with Apache Maven

Apache Maven (Maven) is a software project management and comprehension tool. Based on the concept of a project object model (POM), Maven can manage a project's build, reporting and documentation from a central piece of information. Why Maven? Let’s learn about the Maven’s history here https://maven.apache.org/background/history-of-maven.html to understand why Maven is created. There are a lot of useful things that we should use Maven. In my opinion, the most important thing is that Maven solves the problems with managing jar files. It centralizes these files in one place and it is easy to use by declaring dependencies in a xml file (pom.xml). Using Maven to manage JAR files Firstly, I would like to give you an example about using some JAR files without Maven. In my project, I need some JAR files such as SNMP4J.jar, jta26.jar, jgraphx.jar, etc… so I had to search them on the Internet with visiting a lot of websites, download and add these files into my proje...

vhandit

Search This Blog