Improving the execution time of CI pipelines

Executing a large number of tests, especially integration tests, takes a lot of time. For instance, the pipeline of one of our projects for each Pull Request previously took nearly 30 minutes, including over 1 thousand test cases. This article guides you through several good techniques that we have discovered and applied to improve the time-consuming process.

Parallel stages

Analyze the current phases in your pipeline and categorize them in parallel. For example, we can separate the build and verify code of Node.js and Maven modules simultaneously in our Jenkins pipelines. Please mind using the setting failFast whether you want to abort the pipeline immediately.

Parallel test execution

If you use Maven, the plugin maven-failsafe-plugin is used to execute integration tests during phases integration-test and verify the build lifecycle. It allows us to execute tests in parallel. There are many settings related to parallel test execution, so make sure to configure them correctly for your environment. For example, we can execute tests in parallel using the classes mode and set threadCount=7 for the Jenkins Agent with 8 CPU cores.

Application profiles

There are certain integration tests that don't require the use of a database. These tests focus on scenarios like error handling, authentication filters, or loading configuration. For example, initializing the database with Testcontainers is time-consuming. If you use Spring Boot, you can utilize a different profile and use the H2 in-memory database for these tests. The test class will be annotated with the @ActiveProfiles annotation to run with the specified profile.

Comments

How to apply Lean - Kanban for your business

This is the topic of Scrum Breakfast meetup this time, speaker: Ms. Phuong Bui - Technical Project Manager of YOOSE Pte. Ltd. http://www.meetup.com/Scrum-Breakfast-Vietnam-Agile-and-Scrum-Meetup/events/230313727/ Lean comes from Lean manufacturing is a method that focuses on elimination of wastes. In other words, this is a set of principles for archiving the quality, speed and customer alignment. The first time I knew about the term "Lean" is from the book Software Craftsmanship . Sandro recommends if we want to transform our pet projects into a real business, we should get familiar with Lean Startup concepts. In this talk, Ms. Phuong pointed out some major wastes includes information (ex: unclear requirements), processes (ex: waiting), physical environment and people. Knowing what the problems should be the best way to eliminate them. The difference between Single item flow and Batch processing is the second main point; and it is the Lean's idea. Batch pr...

Adding a default text to the beginning of an text area

I have a text area where I can enter a SQL statement. The text "SELECT TOP 1" is used at the begining of the sql statement as a default prefix that can not be removed. In order to archive this functionality, I found three options: 1. Use an inputMask and build a pattern for the entered text. We can use the prefix inside the text area: http://jsfiddle.net/SEXAj/2130/ 2. Create an javascript event listener on event "input" and calculate to prevent the prefix is removed http://jsfiddle.net/sarbbottam/TUc94/ 3. Use another html tag like "div"/"span" that contains the prefix and calculate the suitable position + Tag "span": http://jsfiddle.net/4YzZy/ + Tag "div": http://jsfiddle.net/215b34fs/ I chose the 3rd approach with tag "div" because it's the most easiest solution and suitable for me at that monment. References: [1]. http://stackoverflow.com/questions/24846041/how-do-i-add-a-default-text...

The HelloWorld example of JSF 2.2 with Myfaces

I just did by myself create a very simple app "HelloWorld" of JSF 2.2 with a concrete implementation Myfaces that we can use it later on for our further JSF trying out. I attached the source code link at the end part. Just follow these steps below: 1. Create a Maven project in Eclipse (Kepler) with a simple Java web application archetype "maven-archetype-webapp". Maven should be the best choice for managing the dependencies , so far. JSF is a web framework that is the reason why I chose the mentioned archetype for my example. 2. Import dependencies for JSF implementation - Myfaces (v2.2.10) into file pom.xml . The following code that is easy to find from http://mvnrepository.com/ with key words "myfaces". <dependency> <groupId>org.apache.myfaces.core</groupId> <artifactId>myfaces-api</artifactId> <version>2.2.10</version> </dependency> <dependency> <groupId>org.apache.myfaces.core<...

My must-have apps for daily work

There is no doubt that cool apps can help us be more productive and enjoyable at work. For the time being, I really love the following apps which are used by me almost every day. 1. A personal Kanban In fact, a personal kanban is the most useful app for me. Why does it matter? It is not just a to-do list, but it keeps me motivated every day because it helps me be able to know what my "big picture" is. I usually set up my plans together with a path to reach them. KanbanFlow is my preferred tool. KanbanFlow 2. A terminal Needless to say, a terminal is a must-have app for every developer, especially the ones use macOS/Linux. Due to its importance, I love to decorate and enhance it to be super exciting with various tools such as iTerm , oh-my- zsh , and thefuck . ;) iTerm + oh-my-zsh 3. A documentation "ecosystem" As a developer, I can not remember all things that I have experimented a day. Moreover, a document is really useful for sharing an...

Monday vhandit #1

This is the first post in a series of articles called "Monday vhandit". You can go to the original post by clicking on each head title. Character encodings: essential concepts This article points out why Unicode matters and differentiates among concepts: character sets, coded character set (unit is code points), and character encoding. Explained from First Principles: the Internet The very structural and well-explained article contains almost concepts of the Internet. I really appreciated the author's effort put on that post. Dev Community - DevDojo I just joined this community. DevDojo is really a developer-oriented platform. It is very nice, easy, fast, and sufficient to write (Markdown supported) and search for a post. The best of it, you also have a personal blog for your posts at "<your_username>.devdojo.com". My username is vnnvanhuong Everything you need to know about HTTP security headers A good collection of HTTP security-related headers. Moreover...

vhandit

Search This Blog