Optimize Autoware Universe CI/CD For Faster Performance

Autoware Universe, a leading open-source autonomous driving software platform, relies heavily on Continuous Integration and Continuous Delivery (CI/CD) to ensure code quality and accelerate development. However, the build and test process, especially the build-and-test-differential job, can be time-consuming, particularly when modifications are made to packages with numerous dependencies. This article delves into the challenges of Autoware Universe's CI/CD pipeline and proposes solutions to expedite the process by leveraging build cache mechanisms.

Understanding the Challenge

Before diving into solutions, it's crucial to understand the problem. Autoware Universe mandates that the build-and-test-differential job passes before any code merge. This job builds and tests only the changes introduced in a pull request and their direct dependencies. While this approach is efficient for smaller changes, it can become a bottleneck when a modification affects a core package with a large dependency tree. The CI system must rebuild and retest a significant portion of the codebase, leading to extended wait times for developers.

Keywords like CI/CD performance and build time optimization are central to this discussion. Reducing CI/CD time directly translates to faster development cycles and quicker integration of new features and bug fixes. This is especially crucial in a rapidly evolving field like autonomous driving, where agility is key.

The existing process often involves rebuilding dependencies even if they haven't changed. This redundancy wastes valuable CI resources and developer time. By implementing a caching mechanism, we can store the build artifacts of these dependencies and reuse them in subsequent builds, significantly reducing the overall build time.

This article explores two primary approaches to tackle this challenge: creating a dedicated workflow for building and caching dependencies and modifying the existing build-and-test-differential workflow to utilize this cache. Each approach has its own set of advantages and disadvantages, which we will discuss in detail.

Possible Approaches for Autoware Universe CI/CD Optimization

To address the challenge of slow CI/CD times in Autoware Universe, we propose two key strategies focused on leveraging build caches to reduce redundant build processes. These approaches aim to optimize the build-and-test-differential workflow, which is often the bottleneck in the development pipeline. By implementing these solutions, we can significantly speed up the CI/CD pipeline, leading to faster development cycles and more efficient resource utilization. The core idea revolves around identifying and caching dependencies that don't change frequently, thereby avoiding unnecessary rebuilds.

1. Creating a Dedicated Workflow for Dependency Caching

The first approach involves establishing a separate workflow dedicated to building and caching the underlying dependencies of Autoware Universe. This workflow, termed build_dependent_workspace, focuses on creating a Colcon workspace (referred to as dependency_ws) specifically for the packages listed in the build_depends.repos file. The build_depends.repos file essentially outlines the external dependencies required by the Autoware Universe project.

The workflow operates by first setting up a clean environment, installing the necessary dependencies, and then building the packages defined in build_depends.repos within the dependency_ws. The resulting build artifacts, which include compiled libraries, headers, and other necessary files, are then cached and stored in GitHub's caching infrastructure. This cache acts as a repository of pre-built dependencies, ready to be used by other workflows.

The build_dependent_workspace workflow can be configured to run on a scheduled basis, such as daily, or triggered manually whenever there are changes to the build_depends.repos file. This ensures that the cache remains relatively up-to-date with the project's dependencies. The key benefit here is dependency caching, which allows subsequent builds to reuse these pre-built artifacts, significantly reducing build times. By pre-building and caching these dependencies, we avoid the need to rebuild them every time a change is made in the main codebase, thus optimizing build time.

The frequency of running this workflow is a critical consideration. Running it too often may waste resources if dependencies haven't changed, while running it too infrequently may lead to the cache becoming stale. A balance needs to be struck based on the frequency of dependency updates.

2. Modifying the build-and-test-differential Workflow

The second approach focuses on modifying the existing build-and-test-differential workflow to utilize the cached dependencies created by the build_dependent_workspace workflow. This involves incorporating steps into the build-and-test-differential workflow that retrieve and source the dependency_ws before initiating the Colcon build and test processes. Sourcing the dependency_ws essentially makes the pre-built dependencies available to the current build environment.

By sourcing the cached dependencies, the build-and-test-differential workflow can skip the build steps for those dependencies, focusing instead on building and testing only the changed packages and their direct, non-cached dependencies. This selective building approach drastically reduces the overall build time, as the majority of the dependencies are already pre-built and available in the cache. This approach directly improves CI/CD performance by reducing the time spent on building unchanged components.

This modification requires careful integration to ensure that the cached dependencies are correctly sourced and used during the build process. The workflow needs to be updated to handle the retrieval of the cached artifacts from GitHub's cache and to set up the environment correctly so that Colcon can find and use these pre-built dependencies. This CI/CD optimization strategy leverages the cached dependencies to accelerate the build process, making the build-and-test-differential workflow significantly faster. The key here is to ensure that the workflow correctly identifies and utilizes the cached dependencies, avoiding any potential conflicts or inconsistencies.

Defining the Definition of Done

To ensure the successful implementation of these approaches, we define clear