Improving the execution time of CI pipelines

Executing a large number of tests, especially integration tests, takes a lot of time. For instance, the pipeline of one of our projects for each Pull Request previously took nearly 30 minutes, including over 1 thousand test cases. This article guides you through several good techniques that we have discovered and applied to improve the time-consuming process.

Parallel stages

Analyze the current phases in your pipeline and categorize them in parallel. For example, we can separate the build and verify code of Node.js and Maven modules simultaneously in our Jenkins pipelines. Please mind using the setting failFast whether you want to abort the pipeline immediately.

Parallel test execution

If you use Maven, the plugin maven-failsafe-plugin is used to execute integration tests during phases integration-test and verify the build lifecycle. It allows us to execute tests in parallel. There are many settings related to parallel test execution, so make sure to configure them correctly for your environment. For example, we can execute tests in parallel using the classes mode and set threadCount=7 for the Jenkins Agent with 8 CPU cores.

Application profiles

There are certain integration tests that don't require the use of a database. These tests focus on scenarios like error handling, authentication filters, or loading configuration. For example, initializing the database with Testcontainers is time-consuming. If you use Spring Boot, you can utilize a different profile and use the H2 in-memory database for these tests. The test class will be annotated with the @ActiveProfiles annotation to run with the specified profile.

Comments

Books Read in 2022

My bookshelf Technical - Operating System: Three Easy Pieces. Andrea Arpaci-Dusseau, Remzi Arpaci-Dusseau. - The Effective Engineer. Edmond Lau. - English Grammar in Use. Raymond Murphy - Mythical Man-Month, The: Essays on Software Engineering. Brooks Jr., Frederick P. - Peopleware: Productive Projects and Teams (3rd edition). Tom DeMarco, Timothy Lister. - A Philosophy of Software Design. John Ousterhout. - Modern Computer Architecture and Organization: Learn x86, ARM, and RISC-V architectures and the design of smartphones, PCs, and cloud servers. Jim Ledin. Thought Provoking - Đúng Việc. Giản Tư Trung. - A Brief History of Time. Stephen Hawking. - Một đời như kẻ tìm đường. Phan Văn Trường. - Một đời thương thuyết. Phan Văn Trường. - Tinh hoa xử thế - Dám bị ghét. Kishimi Ichiro, Koga Fumitake. - Phép màu để vượt lên chính mình. Nhan Húc Quân. - Scouting for Boys: A Handbook for Instruction in Good Citizenship Through Woodcraft. Robert Baden-Powell. - Thay Đổi Cuộc Sống Với Nhân ...

A Template for Software Engineering Standards

Software engineering standard template A well-structured standard acts as a blueprint that guides engineers in their daily tasks and long-term goals. Below, I will outline a template for creating a comprehensive software engineering standard. Header The header serves as the document's identifier. It contains the following: Authors : The people who have contributed to the creation of the standard. Created Date : The date when the document was initially created. Version : The version of the standard. It is typically updated with significant changes. Status : The current status of the document, whether it's in draft, in-review, or official. Next Review Date : The date when the standard will be reviewed for relevancy and accuracy. Table of Contents A table of contents provides an overview of what the document contains, making it easier for readers to navigate through the document. Body The body of the standard comprises: Values : The core beliefs that guide the decision-maki...

Think like a Engineering Manager

Off-boarding Members can leave the company for various reasons, and as a manager, it is important to take action. Hoping for the best is not a strategy. In the case of a low-performing member, I can kindly issue an official warning, set clear objectives for improvement, and re-evaluate the results. If there is a conflict between members, I need to be mindful and go beyond the situation to list our expectations with corresponding actions. Finally, if a member has a big chance to grow at another company, I can have an honest discussion with that member about the trade-offs. Balance at Work As an engineering manager, it is important to balance involvement in meetings and getting your hands dirty on some topics. The goal is to become a companion to teams. Here are my two actions to deal with the situation: Dedicate time for important-but-not-urgent tasks and prioritize them daily. Categorize work into four lines including management, project support, OKRs, and self-study. Management Conduc...

Journal: This Month I Learned (2023-April)

I learned that it's important to take breaks and practice self-care to avoid burnout. It's also important to communicate with colleagues and managers about workload and prioritize tasks accordingly. Finally, getting hands-on experience with different aspects of the business can be valuable for personal and professional growth. OKRs review The results of the OKR review should not directly impact performance appraisals. Key Results should be straightforward measurements of Objectives, not just a to-do list. They should also be related to daily work, rather than separate topics. There are two ways to review objectives: "Completion" or "Rating". The status should be balanced among objectives and used to adapt to current efforts. Member reassignment How to find a good fit between business and personnel growth: Salary costs increase year by year. It doesn't make sense to increase customer billing year by year. Burnout counter: Engagement I engage to work close...

JSF 2 - Dynamically manipulating the component tree with system events

Let's suppose we want to modify the metadata (attributes) of elements such as render , requried , maxlength but we do not define in JSF tags. The manipulating components can be conducted in Drools files, for example. How could we do? I think that is what we need to change something of component tree during JSF life-cycle. JSF supports event handling throughout the JSF life-cycle. In this post, I use two events: postAddToView for scanning components tree and preRenderView for manipulating the meta of components before rendering to GUI. I modified my own project from previous post for this example. This is my first further JSF trying out with the project as I said before. :) We define the tags f:event below the form - a container component of the components which we want to work on. The valid values for the attribute type for f:event can be found from tag library document of JSF 2. <!DOCTYPE html> <html xmlns="http://www.w3.org/1999/xhtml" x...

vhandit

Search This Blog