Improving the execution time of CI pipelines

Executing a large number of tests, especially integration tests, takes a lot of time. For instance, the pipeline of one of our projects for each Pull Request previously took nearly 30 minutes, including over 1 thousand test cases. This article guides you through several good techniques that we have discovered and applied to improve the time-consuming process.

Parallel stages

Analyze the current phases in your pipeline and categorize them in parallel. For example, we can separate the build and verify code of Node.js and Maven modules simultaneously in our Jenkins pipelines. Please mind using the setting failFast whether you want to abort the pipeline immediately.

Parallel test execution

If you use Maven, the plugin maven-failsafe-plugin is used to execute integration tests during phases integration-test and verify the build lifecycle. It allows us to execute tests in parallel. There are many settings related to parallel test execution, so make sure to configure them correctly for your environment. For example, we can execute tests in parallel using the classes mode and set threadCount=7 for the Jenkins Agent with 8 CPU cores.

Application profiles

There are certain integration tests that don't require the use of a database. These tests focus on scenarios like error handling, authentication filters, or loading configuration. For example, initializing the database with Testcontainers is time-consuming. If you use Spring Boot, you can utilize a different profile and use the H2 in-memory database for these tests. The test class will be annotated with the @ActiveProfiles annotation to run with the specified profile.

Comments

Creating a Chatbot with RiveScript in Java

Motivation "Artificial Intelligence (AI) is considered a major innovation that could disrupt many things. Some people even compare it to the Internet. A large investor firm predicted that some AI startups could become the next Apple, Google or Amazon within five years" - Prof. John Vu, Carnegie Mellon University. Using chatbots to support our daily tasks is super useful and interesting. In fact, "Jenkins CI, Jira Cloud, and Bitbucket" have been becoming must-have apps in Slack of my team these days. There are some existing approaches for chatbots including pattern matching, algorithms, and neutral networks. RiveScript is a scripting language using "pattern matching" as a simple and powerful approach for building up a Chabot. Architecture Actually, it was flexible to choose a programming language for the used Rivescript interpreter like Java, Go, Javascript, Python, and Perl. I went with Java. Used Technologies and Tools Oracle JDK 1.8...

Strategy Design Pattern

For example, I have a program with an Animal abstract class and two sub-classes Dog and Bird. I want to add new behavior for the class Animal, this is "fly". Now, I face two approaches to solve this issue: 1. Adding an abstract method "fly" into the class Animal. Then, I force the sub-classes should be implemented this method, something like: public abstract class Animal{ //bla bla public abstract void fly(); } public class Bird extends Animal{ //bla bla public void fly(){ System.out.println("Fly high"); } } public class Dog extends Animal{ //bla bla public void fly(){ System.out.println("Cant fly"); } } 2. Creating an interface with method "fly" inside. The same issue to an abstract class, I force the classes these implement this interface should have a method "fly" inside: public interface Flyable{ public void fly(); } public class Bird implements Flyable{ //bla bla public void fly(){ System.out.pr...

My must-have apps for daily work

There is no doubt that cool apps can help us be more productive and enjoyable at work. For the time being, I really love the following apps which are used by me almost every day. 1. A personal Kanban In fact, a personal kanban is the most useful app for me. Why does it matter? It is not just a to-do list, but it keeps me motivated every day because it helps me be able to know what my "big picture" is. I usually set up my plans together with a path to reach them. KanbanFlow is my preferred tool. KanbanFlow 2. A terminal Needless to say, a terminal is a must-have app for every developer, especially the ones use macOS/Linux. Due to its importance, I love to decorate and enhance it to be super exciting with various tools such as iTerm , oh-my- zsh , and thefuck . ;) iTerm + oh-my-zsh 3. A documentation "ecosystem" As a developer, I can not remember all things that I have experimented a day. Moreover, a document is really useful for sharing an...

Git Feature Branch Workflow

Motivator It's important for a team to have an agreement on how the changes of source code should be applied. According to projects and teams size, we will define a workflow or select one from recommended workflows ; the "Feature Branch Workflow" is a candidate. What is it? - One branch "master" for main codebase - Several separated branches for features development Why should we care? - Be super simple and allow each developer works on a particular feature. - A stable codebase (master) benefits for continuous integration (CI) environment - Leverage "Pull request" for Code review How it works? A lifecyle of a feature branch (usually created by a story) 1. Creator creates a new branch from a story. For example: "ABC-1-setup-projects" 2. Creator checkouts the created branch and works on the branch (commits, pushes) 3. Creator has done the feature, he uses "pull request" to merge his branch into branch "master...

Updates to the Scrum Guide - The 5 Scrum Values

This article is available at blog.scrum.org , here I just quote my favorite points and give my comment at the end of this post. Ken Schwaber and Jeff Sutherland, the creators of Scrum delivered a webinar on their latest update to the Scrum Guide . The update was a simple one, adding the 5 values of Scrum to the Guide: These values sound easy? Well, there are many misunderstandings and common problems when applying these values. Here are some example: Value Misunderstanding Getting the value right Commitment Committing to something that you don’t understand because you are told to by your boss. Committing yourself to the team and Sprint Goal. Focus Focusing on keeping the customer happy Being focused on the sprint and its goal. Openness Telling everyone everything about all your work Highlighting when you have challenges and problems that are stopping you from success Respect Thinking you are helping the team by being a hero Helping peop...

vhandit

Search This Blog